--- title: Pipelines Customisation slug: klevu-magento-v4/entities-pipelines-customisation docTags: createdAt: 2024-08-06T16:09:24.308Z --- Making customisations to existing Klevu pipelines. :::hint{type="warning"} Before adding attributes to your pipeline YAML, please ensure they have been [registered with Klevu](https://docs.klevu.com/klevu-magento-v4/attributes#DaeAe). ::: :::hint{type="info"} Note: any customisations made via these methods will override the auto-generated pipeline configuration files found in `var/klevu/indexing/pipeline` During development, it may be useful to edit (in a non-production environment) these auto-generated YAML files to test out customisations before copying the YAML to a custom override file. These files will be regenerated and can be deleted once the customisation override has been created. ::: ## Adding Pipeline Stages :::hint{type="info"} Note: All these examples could be combined into one `pipelineConfigurationOverrideFilepaths` file. They have been split into multiple files, as we do in the modules, to make them easier to read. ::: ### Example 1: Add `meta_description` and `meta_title` to the default (simple, virtual, downloadable product types) product data during the Add action. First, in your module add or update `etc/di.xml`. Add a virtual type for `Klevu\IndexingProducts\Service\Provider\PipelineConfigurationOverridesFilepathsProvider\Add` and pass in an array of `pipelineConfigurationOverrideFilepaths` :::CodeblockTabs Vendor/Module/etc/di.xml ```xml Vendor_Module::etc/pipeline/add_update_default.yml ``` ::: If the same changes are required during an Update (almost always the case) then we’d also make the same change to `Klevu\IndexingProducts\Service\Provider\PipelineConfigurationOverridesFilepathsProvider\Update` :::CodeblockTabs Vendor/Module/etc/di.xml ```xml Vendor_Module::etc/pipeline/add_update_default.yml ``` ::: Create the YAML file. :::CodeblockTabs Vendor/Module/etc/pipeline/add\_update\_default.yml ```yaml stages: iterateIndexingRecordsBatch: stages: iterateIndexingRecords: stages: processProduct: stages: default: # Product type stages: generateRecord: stages: attributes: addStages: # add attribute data to the existing pipeline # add stages accepts an array of stages to be added meta_description: import: Vendor_Module::etc/pipeline/product/attributes/meta_description.yml meta_title: import: Vendor_Module::etc/pipeline/product/attributes/meta_title.yml ``` ::: And add the 2 referenced file (this code could also be added to the above YAML, but it is more readable when split into multiple files) :::CodeblockTabs Vendor/Module/etc/pipeline/product/attributes/meta\_description.yml ```yaml stages: getValue: pipeline: Stage\Extract args: extraction: currentProduct::getMetaDescription() transformations: - StripTags(null, ["script"]) - EscapeHtml - toString ``` ::: :::CodeblockTabs Vendor/Module/etc/pipeline/product/attributes/meta\_title.yml ```yaml stages: getValue: pipeline: Stage\Extract args: extraction: currentProduct::getMetaTitle() transformations: - EscapeHtml - toString ``` ::: ### Example 2: Add `color` and `size` to configurable variant product data during the sync action. First, in your module add or update `etc/di.xml`. Add a virtual type `Klevu\IndexingProducts\Service\Provider\PipelineConfigurationOverridesFilepathsProvider\Add` and pass in an array of `pipelineConfigurationOverrideFilepaths` :::CodeblockTabs Vendor/Module/etc/di.xml ```xml Vendor_Module::etc/pipeline/add_update_variant.yml ``` ::: If the same changes are required during an Update then we’d also make this change to `Klevu\IndexingProducts\Service\Provider\PipelineConfigurationOverridesFilepathsProvider\Update` :::CodeblockTabs Vendor/Module/etc/di.xml ```xml Vendor_Module::etc/pipeline/add_update_variant.yml ``` ::: Create the YAML file. :::CodeblockTabs Vendor/Module/etc/pipeline/add\_update\_variant.yml ```yaml stages: iterateIndexingRecordsBatch: stages: iterateIndexingRecords: stages: processProduct: stages: variantProduct: # Product type stages: generateRecord: stages: attributes: addStages: # add attribute data to the existing pipeline # add stages accepts an array of stages to be added color: import: Vendor_Module::etc/pipeline/product/attributes/color.yml size: import: Vendor_Module::etc/pipeline/product/attributes/size.yml ``` ::: And add the 2 referenced file (this code could also be added to the above YAML, but it is more readable when split into multiple files) :::CodeblockTabs Vendor/Module/etc/pipeline/product/attributes/color.yml ```yaml stages: getValue: pipeline: Stage\Extract args: extraction: currentProduct:: transformations: - GetAttributeText("color") - ToString - Trim ``` ::: :::CodeblockTabs Vendor/Module/etc/pipeline/product/attributes/color.yml ```yaml stages: getValue: pipeline: Stage\Extract args: extraction: currentProduct:: transformations: - GetAttributeText("size") - ToString - Trim ``` ::: *** ## Updating Existing Pipeline Stages This works the same as adding a new pipeline stage. In this example, we will append extra data to the `description`. First, in your module add or update `etc/di.xml`. Add a virtual type `Klevu\IndexingProducts\Service\Provider\PipelineConfigurationOverridesFilepathsProvider\Add` and pass in an array of `pipelineConfigurationOverrideFilepaths` :::CodeblockTabs Vendor/Module/etc/di.xml ```xml Vendor_Module::etc/pipeline/add_update_default.yml ``` ::: If the same changes are required during an Update then we’d also make this change to `Klevu\IndexingProducts\Service\Provider\PipelineConfigurationOverridesFilepathsProvider\Update` :::CodeblockTabs Vendor/Module/etc/di.xml ```xml Vendor_Module::etc/pipeline/add_update_default.yml ``` ::: Create the YAML file. :::CodeblockTabs Vendor/Module/etc/pipeline/add\_update\_default.yml ```yaml stages: iterateIndexingRecordsBatch: stages: iterateIndexingRecords: stages: processProduct: stages: default: # Product type stages: generateRecord: stages: attributes: addStages: # can also be used to override existing stage definitions description: import: Vendor_Module::etc/pipeline/product/attributes/description.yml ``` ::: Append the product name and sku to the end of the description :::CodeblockTabs Vendor/Module/etc/pipeline/product/attributes/description.yml ```yaml stages: getValue: pipeline: Stage\Extract args: extraction: currentProduct::getDescription() transformations: - Append(" - ", $currentProduct::getName(), " - ", $currentProduct::getSku()) - StripTags(null, ["script"]) - EscapeHtml ``` ::: *** ## Removing Pipeline Stages In this example, we will remove the `rating` and `rating count` from any configurable variant so indexing falls back to using the parent values First, in your module add or update `etc/di.xml`. Add a virtual type `Klevu\IndexingProducts\Service\Provider\PipelineConfigurationOverridesFilepathsProvider\Add` and pass in an array of `pipelineConfigurationOverrideFilepaths` :::CodeblockTabs Vendor/Module/etc/di.xml ```xml Vendor_Module::etc/pipeline/add_update_variant.yml ``` ::: If the same changes are required during an Update then we’d also make this change to `Klevu\IndexingProducts\Service\Provider\PipelineConfigurationOverridesFilepathsProvider\Update` :::CodeblockTabs Vendor/Module/etc/di.xml ```xml Vendor_Module::etc/pipeline/add_update_variant.yml ``` ::: Create the injected customisation YAML file :::CodeblockTabs Vendor/Module/etc/pipeline/add\_update\_variant.yml ```yaml stages: iterateIndexingRecordsBatch: stages: iterateIndexingRecords: stages: processProduct: stages: variantProduct: # Product type stages: generateRecord: stages: attributes: removeStages: # remove attribute data from the existing pipeline # remove stages accepts an array of stages to be removed - rating - ratingCount ``` ::: *** ## Example Module An example module with YAML files for common attributes that could be useful. Contains examples of how to: - change data synced for existing attributes - add stages to pipelines - add different attributes for different products types. - add new product types - remove attributes - remove attributes from data if some condition is met (in this case if the value is null) List of attribute YAML files present in the module - `color` - `gender` - `manufacturer` - `meta_description` - `meta_keywords` - `meta_title` - `new` - `sale` - `size` - `weight` ::File{src="https://api.archbee.com/api/presign/Y9Igf1t4Mq0XfqFm13RFY-jKmL7VokEtV1f07JZnak0-20241203-170853.zip" label="Example_CustomisePipelineData.zip"} :::hint{type="danger"} This is not a production ready module and is intended as an example only. **Do not install this module in a production environment.** :::