Helper class to create VPRecipies from IR instructions. More...

#include "Transforms/Vectorize/VPRecipeBuilder.h"

Public Member Functions
	VPRecipeBuilder (VPlan &Plan, const TargetLibraryInfo TLI, LoopVectorizationLegality Legal, LoopVectorizationCostModel &CM, VPBuilder &Builder)
VPRecipeBase *	tryToCreateWidenNonPhiRecipe (VPSingleDefRecipe *R, VFRange &Range)
	Create and return a widened recipe for a non-phi recipe `R` if one can be created within the given VF `Range`.
VPRecipeBase *	tryToWidenMemory (VPInstruction *VPI, VFRange &Range)
	Check if the load or store instruction `VPI` should widened for `Range.Start` and potentially masked.
VPHistogramRecipe *	widenIfHistogram (VPInstruction *VPI)
	If `VPI` represents a histogram operation (as determined by LoopVectorizationLegality) make that safe for vectorization, by emitting a llvm.experimental.vector.histogram.add intrinsic in place of the Load + Add\|Sub + Store operations that perform the histogram in the original scalar loop.
bool	replaceWithFinalIfReductionStore (VPInstruction *VPI, VPBuilder &FinalRedStoresBuilder)
	If `VPI` is a store of a reduction into an invariant address, delete it.
void	setRecipe (Instruction I, VPRecipeBase R)
	Set the recipe created for given ingredient.
VPRecipeBase *	getRecipe (Instruction *I)
	Return the recipe created for given ingredient.
VPReplicateRecipe *	handleReplication (VPInstruction *VPI, VFRange &Range)
	Build a VPReplicationRecipe for `VPI`.
VPValue *	getVPValueOrAddLiveIn (Value *V)

Detailed Description

Helper class to create VPRecipies from IR instructions.

Definition at line 26 of file VPRecipeBuilder.h.

Constructor & Destructor Documentation

◆ VPRecipeBuilder()

llvm::VPRecipeBuilder::VPRecipeBuilder	(	VPlan &	Plan,
		const TargetLibraryInfo *	TLI,
		LoopVectorizationLegality *	Legal,
		LoopVectorizationCostModel &	CM,
		VPBuilder &	Builder )

inline

Definition at line 66 of file VPRecipeBuilder.h.

Member Function Documentation

◆ getRecipe()

VPRecipeBase * llvm::VPRecipeBuilder::getRecipe ( Instruction * I )

inline

Return the recipe created for given ingredient.

Definition at line 104 of file VPRecipeBuilder.h.

References assert(), and I.

Referenced by llvm::VPlanTransforms::createInterleaveGroups().

◆ getVPValueOrAddLiveIn()

VPValue * llvm::VPRecipeBuilder::getVPValueOrAddLiveIn ( Value * V )

inline

Definition at line 117 of file VPRecipeBuilder.h.

References llvm::dyn_cast(), and I.

Referenced by widenIfHistogram().

◆ handleReplication()

VPReplicateRecipe * VPRecipeBuilder::handleReplication	(	VPInstruction *	VPI,
		VFRange &	Range )

Build a VPReplicationRecipe for VPI.

If it is predicated, add the mask as last operand. Range.End may be decreased to ensure same recipe behavior from Range.Start to Range.End.

Definition at line 6807 of file LoopVectorize.cpp.

References assert(), llvm::cast(), llvm::dbgs(), llvm::VPRecipeBase::getDebugLoc(), llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), getIntrinsicID(), llvm::VPInstruction::getMask(), llvm::VPSingleDefRecipe::getUnderlyingInstr(), I, llvm::isa(), LLVM_DEBUG, llvm::VPInstruction::operandsWithoutMask(), and Range.

Referenced by llvm::VPlanTransforms::makeMemOpWideningDecisions().

◆ replaceWithFinalIfReductionStore()

bool VPRecipeBuilder::replaceWithFinalIfReductionStore	(	VPInstruction *	VPI,
		VPBuilder &	FinalRedStoresBuilder )

If VPI is a store of a reduction into an invariant address, delete it.

If it is the final store of a reduction result, a uniform store recipe will be created for it in the middle block. Returns true if replacement took place. The order of stores must be preserved, hence FinalRedStoresBuidler.

Definition at line 6788 of file LoopVectorize.cpp.

References llvm::dyn_cast(), llvm::VPRecipeBase::eraseFromParent(), llvm::VPRecipeBase::getDebugLoc(), llvm::VPSingleDefRecipe::getUnderlyingInstr(), llvm::VPBuilder::insert(), and llvm::VPInstruction::operandsWithoutMask().

Referenced by llvm::VPlanTransforms::makeMemOpWideningDecisions().

◆ setRecipe()

void llvm::VPRecipeBuilder::setRecipe	(	Instruction *	I,
		VPRecipeBase *	R )

inline

Set the recipe created for given ingredient.

Definition at line 97 of file VPRecipeBuilder.h.

References assert(), and I.

Referenced by llvm::VPlanTransforms::makeMemOpWideningDecisions().

◆ tryToCreateWidenNonPhiRecipe()

VPRecipeBase * VPRecipeBuilder::tryToCreateWidenNonPhiRecipe	(	VPSingleDefRecipe *	R,
		VFRange &	Range )

Create and return a widened recipe for a non-phi recipe R if one can be created within the given VF Range.

Definition at line 6872 of file LoopVectorize.cpp.

References assert(), llvm::cast(), llvm::VPRecipeBase::getDebugLoc(), llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::Instruction::getOpcode(), llvm::VPInstruction::getOpcode(), llvm::VPUser::getOperand(), llvm::is_contained(), llvm::Instruction::isCast(), llvm::ElementCount::isScalar(), llvm::VPInstruction::operandsWithoutMask(), and Range.

◆ tryToWidenMemory()

VPRecipeBase * VPRecipeBuilder::tryToWidenMemory	(	VPInstruction *	VPI,
		VFRange &	Range )

Check if the load or store instruction VPI should widened for Range.Start and potentially masked.

Such instructions are handled by a recipe that takes an additional VPInstruction for the mask.

Definition at line 6484 of file LoopVectorize.cpp.

References assert(), llvm::cast(), llvm::LoopVectorizationCostModel::CM_Interleave, llvm::LoopVectorizationCostModel::CM_Scalarize, llvm::LoopVectorizationCostModel::CM_Unknown, llvm::LoopVectorizationCostModel::CM_Widen, llvm::LoopVectorizationCostModel::CM_Widen_Reverse, llvm::dyn_cast(), GEP, llvm::VPRecipeBase::getDebugLoc(), llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::getLoadStoreType(), llvm::VPInstruction::getMask(), llvm::VPInstruction::getOpcode(), llvm::VPUser::getOperand(), llvm::VPSingleDefRecipe::getUnderlyingInstr(), llvm::VPValue::getUnderlyingValue(), I, llvm::GEPNoWrapFlags::none(), Range, llvm::Reverse, llvm::VPInstruction::Reverse, and llvm::Value::stripPointerCasts().

Referenced by llvm::VPlanTransforms::makeMemOpWideningDecisions().

◆ widenIfHistogram()

VPHistogramRecipe * VPRecipeBuilder::widenIfHistogram ( VPInstruction * VPI )

If VPI represents a histogram operation (as determined by LoopVectorizationLegality) make that safe for vectorization, by emitting a llvm.experimental.vector.histogram.add intrinsic in place of the Load + Add|Sub + Store operations that perform the histogram in the original scalar loop.

Definition at line 6759 of file LoopVectorize.cpp.

References assert(), llvm::cast(), llvm::VPRecipeBase::getDebugLoc(), llvm::VPInstruction::getMask(), llvm::VPInstruction::getOpcode(), llvm::VPUser::getOperand(), llvm::VPSingleDefRecipe::getUnderlyingInstr(), getVPValueOrAddLiveIn(), and llvm::SmallVectorTemplateBase< T, bool >::push_back().

Referenced by llvm::VPlanTransforms::makeMemOpWideningDecisions().

The documentation for this class was generated from the following files:

lib/Transforms/Vectorize/VPRecipeBuilder.h
lib/Transforms/Vectorize/LoopVectorize.cpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ VPRecipeBuilder()

Member Function Documentation

◆ getRecipe()

◆ getVPValueOrAddLiveIn()

◆ handleReplication()

◆ replaceWithFinalIfReductionStore()

◆ setRecipe()

◆ tryToCreateWidenNonPhiRecipe()

◆ tryToWidenMemory()

◆ widenIfHistogram()