Pipelines

if an attribute has been registered with klevu for indexing that does not mean that the entity data from that attribute is automatically sent to klevu it means it can be sent to klevu entity data indexed to klevu is defined in the indexing pipelines yaml files these files can either be auto generated via the product attribute setting automatically generate pipeline configuration for see https //docs klevu com/klevu magento v4/attributes#voppl manually created if the auto generated code is not desirable, see https //docs klevu com/klevu magento v4/entities pipelines customisation the auto generated pipeline yaml files can be found in var/klevu/indexing/pipeline/ php sdk, pipelines & magento overview klevu indexing uses https //docs klevu com/indexing apis/overview klevu has developed a https //github com/klevu/php sdk to assist developers with integrating klevu indexing into any php platform this php sdk is used to index data to klevu in these magento modules klevu has also developed https //github com/klevu/php pipelines , these are generic php pipelines that can be used for any purpose and are independent of magento and the php sdk these 2 modules are brought together via https //github com/klevu/php sdk pipelines , adding php sdk functionality into the pipelines there is a 4th module, https //github com/klevu/module m2 pipelines which integrates these new components into magento this handles dependency injection and adds some magento context into the pipelines types of pipeline create record pipeline pipeline\createrecord create and return a new object example create an object with keys id and type pipeline pipeline\createrecord stages id pipeline stage\extract args extraction currentproduct getentityid() transformations tostring type pipeline stage\staticvalue args value "klevu product" output { "id" "123", "type" "klevu product" } fallback pipeline pipeline\fallback if any validation exception validationexceptioninterface is thrown, catch it and move on to the next stage, otherwise return the output from this stage example get the product type, validate that the type is “bundle” if valid include this file product/bundle yml if not valid move to the next stage which will include this file product/default yml bundleproduct stages \ pipeline stage\extract args extraction currentproduct gettypeid() \ pipeline stage\validate args validation isequalto("bundle") \ import klevu indexingproducts etc/pipeline/product/bundle yml default stages \ import klevu indexingproducts etc/pipeline/product/default yml note the stage names have been removed from this example to make it easier to read iterate pipeline pipeline\iterate loop over the provided records example iterate over all records provided into the pipeline stages iterateindexingrecords pipeline pipeline\iterate stages \ or get all customer groups, iterate over then, setting the current customer group at the start of each loop getcustomergroups pipeline stage\extract args extraction store getforcurrentstore() customer groups transformations tostring iteratecustomergroups pipeline pipeline\iterate # loop over all customer groups stages setcustomergroupid pipeline stage\setcustomergroupid pipeline stages static value add a value to the output example pipeline pipeline\createrecord stages type pipeline stage\staticvalue args value "klevu product" output { "type" "klevu product" } extract get the value from the extraction and pass it to the next stage example pipeline pipeline\createrecord stages id pipeline stage\extract args extraction currentproduct getentityid() output { "id" 123 } transform transform the provided value using one or multiple transformers example pipeline pipeline\createrecord stages id \ pipeline stage\extract args extraction currentproduct getentityid() \ pipeline stage\transform args transformation tostring|trim|prepend("id ") output { "id" "id 123" } alternatively, transformations can be passed into extractions, skipping the need to add an extra stage example pipeline pipeline\createrecord stages name pipeline stage\extract args extraction currentproduct getname() transformations \ trim \ append(" ", $currentproduct getmanufacturer()) output { "name" "test product brand name" } validate example \ pipeline stage\extract args extraction currentproduct getklevuimage() \ pipeline stage\validate args validation isnotequalto("no selection")|isnotempty if validation fails an exception of type klevu\pipelines\exception\validationexceptioninterface will be thrown fallback pipelines will catch this an proceed to the next stage see module m2 indexing products/etc/pipeline/attributes/images yml register context register a variable to be used elsewhere in the pipeline example extract the current product and then register the context as currentproduct extractproduct pipeline stage\extract args extraction currentindexingrecord getentity() registerproduct pipeline stage\registercontext args contextkey "currentproduct" then later in the pipeline (possibly in another file, even another module) the context can be used pipeline pipeline\createrecord stages default stages \ pipeline stage\extract args extraction currentproduct getname() transformations escapehtml output { default 'product name' } log log the current context example module m2 indexing/etc/pipeline/process batch payload yml take whatever the current context is (the output from the previous operation) and log it along with the message “chunked payload” at log level “debug” stages log pipeline indexing\stage\log args message "chunked payload" level debug output (formatted to make it more readable) report debug chunked payload { "identifier" "processpayload processrecords iterate log", "payload" \[ {"id" "47","type" "klevu product","relations" {"categories" {"values" \["categoryid 15","categoryid 36","categoryid 2","categoryid 12"]}},"attributes" {"name" {"default" "chaz kangeroo hoodie xs black"},"sku" "mh01 xs black","images" \[{"url" "https //magento test/media/catalog/product/cache/7fe2def9e1d815fff5838a5b86f54f8b/m/h/mh01 black main 1 jpg","type" "default","width" 800,"height" 800}],"prices" \[{"amount" 52 0,"type" "defaultprice","currency" "usd"},{"amount" 52 0,"type" "saleprice","currency" "usd"}],"url" "https //magento test/catalog/product/view/id/47/s/chaz kangeroo hoodie xs black/","instock"\ true,"shortdescription" {"default"\ null},"description" {"default" "ideal for cold weather training or work outdoors, the chaz hoodie promises superior warmth with every wear thick material blocks out the wind as ribbed cuffs and bottom band seal in body heat \• two tone gray heather hoodie \• drawstring adjustable hood \• machine wash/dry "},"tags" \[],"visibility" \[],"rating" 4 2,"ratingcount" 321}}, {"id" "2051 2053","type" "klevu product","relations" {"parentproduct" {"values" \["2051"]}},"attributes" {"sku" "configurable product link test orange 1","images"\ null,"type" "default","width" 800,"height" 800}],"prices" \[{"amount" 45 0,"type" "defaultprice","currency" "usd"},{"amount" 45 0,"type" "saleprice","currency" "usd"}],"url" "https //magento test/configurable product link test html","instock"\ true,"shortdescription" {"default"\ null},"description" {"default"\ null},"rating" 4 7,"ratingcount" 123}} ] } \[]