Helper class to create VPRecipies from IR instructions. More...

#include "Transforms/Vectorize/VPRecipeBuilder.h"

Public Member Functions
	VPRecipeBuilder (Loop OrigLoop, const TargetLibraryInfo TLI, LoopVectorizationLegality *Legal, LoopVectorizationCostModel &CM, PredicatedScalarEvolution &PSE, VPBuilder &Builder)

VPRecipeOrVPValueTy	tryToCreateWidenRecipe (Instruction Instr, ArrayRef< VPValue > Operands, VFRange &Range, VPlanPtr &Plan)
	Check if an existing VPValue can be used for `Instr` or a recipe can be create for `I` withing the given VF `Range`. More...

void	setRecipe (Instruction I, VPRecipeBase R)
	Set the recipe created for given ingredient. More...

VPValue *	createBlockInMask (BasicBlock *BB, VPlanPtr &Plan)
	A helper function that computes the predicate of the block BB, assuming that the header block of the loop is set to True. More...

VPValue *	createEdgeMask (BasicBlock Src, BasicBlock Dst, VPlanPtr &Plan)
	A helper function that computes the predicate of the edge between SRC and DST. More...

void	recordRecipeOf (Instruction *I)
	Mark given ingredient for recording its recipe once one is created for it. More...

VPRecipeBase *	getRecipe (Instruction *I)
	Return the recipe created for given ingredient. More...

VPRegionBlock *	createReplicateRegion (Instruction I, VPRecipeBase PredRecipe, VPlanPtr &Plan)
	Create a replicating region for instruction `I` that requires predication. More...

VPBasicBlock *	handleReplication (Instruction I, VFRange &Range, VPBasicBlock VPBB, VPlanPtr &Plan)
	Build a VPReplicationRecipe for `I` and enclose it within a Region if it is predicated. More...

void	fixHeaderPhis ()
	Add the incoming values from the backedge to reduction & first-order recurrence cross-iteration phis. More...

Detailed Description

Helper class to create VPRecipies from IR instructions.

Definition at line 27 of file VPRecipeBuilder.h.

Constructor & Destructor Documentation

◆ VPRecipeBuilder()

llvm::VPRecipeBuilder::VPRecipeBuilder	(	Loop *	OrigLoop,
		const TargetLibraryInfo *	TLI,
		LoopVectorizationLegality *	Legal,
		LoopVectorizationCostModel &	CM,
		PredicatedScalarEvolution &	PSE,
		VPBuilder &	Builder
	)

inline

Definition at line 109 of file VPRecipeBuilder.h.

References Builder, and llvm::IRSimilarity::Legal.

Member Function Documentation

◆ createBlockInMask()

VPValue * VPRecipeBuilder::createBlockInMask	(	BasicBlock *	BB,
		VPlanPtr &	Plan
	)

A helper function that computes the predicate of the block BB, assuming that the header block of the loop is set to True.

It returns the entry mask for the block BB.

Definition at line 8054 of file LoopVectorize.cpp.

References llvm::VPInstruction::ActiveLaneMask, assert(), BB, llvm::InnerLoopVectorizer::Builder, llvm::LoopBase< BlockT, LoopT >::contains(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPBasicBlock::getFirstNonPhi(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::VPInstruction::ICmpULE, llvm::VPBasicBlock::insert(), IV, llvm::InnerLoopVectorizer::OrigLoop, and llvm::predecessors().

◆ createEdgeMask()

VPValue * VPRecipeBuilder::createEdgeMask	(	BasicBlock *	Src,
		BasicBlock *	Dst,
		VPlanPtr &	Plan
	)

A helper function that computes the predicate of the edge between SRC and DST.

Definition at line 8009 of file LoopVectorize.cpp.

References assert(), llvm::InnerLoopVectorizer::Builder, llvm::BranchInst::getCondition(), llvm::Instruction::getDebugLoc(), llvm::ConstantInt::getFalse(), llvm::BranchInst::getSuccessor(), llvm::Value::getType(), llvm::is_contained(), llvm::BranchInst::isConditional(), llvm::LoopBase< BlockT, LoopT >::isLoopExiting(), llvm::InnerLoopVectorizer::OrigLoop, and llvm::predecessors().

◆ createReplicateRegion()

VPRegionBlock * VPRecipeBuilder::createReplicateRegion	(	Instruction *	I,
		VPRecipeBase *	PredRecipe,
		VPlanPtr &	Plan
	)

Create a replicating region for instruction I that requires predication.

PredRecipe is a VPReplicateRecipe holding I.

Definition at line 8478 of file LoopVectorize.cpp.

References assert(), llvm::Instruction::getOpcodeName(), and llvm::Instruction::getParent().

◆ fixHeaderPhis()

void VPRecipeBuilder::fixHeaderPhis ( )

Add the incoming values from the backedge to reduction & first-order recurrence cross-iteration phis.

Definition at line 8386 of file LoopVectorize.cpp.

References llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), llvm::VPDef::getVPSingleValue(), and llvm::InnerLoopVectorizer::OrigLoop.

◆ getRecipe()

VPRecipeBase* llvm::VPRecipeBuilder::getRecipe ( Instruction * I )

inline

Return the recipe created for given ingredient.

Definition at line 152 of file VPRecipeBuilder.h.

References assert(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >, KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >::count(), and I.

◆ handleReplication()

VPBasicBlock * VPRecipeBuilder::handleReplication	(	Instruction *	I,
		VFRange &	Range,
		VPBasicBlock *	VPBB,
		VPlanPtr &	Plan
	)

Build a VPReplicationRecipe for I and enclose it within a Region if it is predicated.

Returns: VPBB augmented with this new recipe if I is not predicated, otherwise; a new VPBasicBlock that succeeds the new Region. Update the packing decision of predicated instructions if they feed I. Range.End may be decreased to ensure same recipe behavior from Range.Start to Range.End.

Definition at line 8396 of file LoopVectorize.cpp.

References llvm::VPBasicBlock::appendRecipe(), assert(), llvm::VPBlockUtils::connectBlocks(), llvm::dbgs(), llvm::VPBlockUtils::disconnectBlocks(), llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), getIntrinsicID(), llvm::VPBlockBase::getSingleSuccessor(), I, llvm::VPBlockUtils::insertBlockAfter(), llvm::LinearPolySize< LeafTy >::isScalable(), LLVM_DEBUG, llvm::VFRange::Start, and llvm::InnerLoopVectorizer::VF.

◆ recordRecipeOf()

void llvm::VPRecipeBuilder::recordRecipeOf ( Instruction * I )

inline

Mark given ingredient for recording its recipe once one is created for it.

Definition at line 145 of file VPRecipeBuilder.h.

References assert(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >, KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >::count(), and I.

◆ setRecipe()

void llvm::VPRecipeBuilder::setRecipe	(	Instruction *	I,
		VPRecipeBase *	R
	)

inline

Set the recipe created for given ingredient.

This operation is a no-op for ingredients that were not marked using a nullptr entry in the map.

Definition at line 126 of file VPRecipeBuilder.h.

References assert(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >, KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >::count(), and I.

◆ tryToCreateWidenRecipe()

VPRecipeOrVPValueTy VPRecipeBuilder::tryToCreateWidenRecipe	(	Instruction *	Instr,
		ArrayRef< VPValue * >	Operands,
		VFRange &	Range,
		VPlanPtr &	Plan
	)

Check if an existing VPValue can be used for Instr or a recipe can be create for I withing the given VF Range.

If an existing VPValue can be used or if a recipe can be created, return it. Otherwise return a VPRecipeOrVPValueTy with nullptr.

Definition at line 8512 of file LoopVectorize.cpp.

References assert(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::find(), GEP, llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::PHINode::getIncomingValueForBlock(), llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), llvm::LoopBase< BlockT, LoopT >::getLoopPreheader(), llvm::Instruction::getParent(), llvm::RecurrenceDescriptor::getRecurrenceStartValue(), llvm::LoopVectorizationLegality::getReductionVars(), llvm::PredicatedScalarEvolution::getSCEV(), llvm::PredicatedScalarEvolution::getSE(), llvm::LoopVectorizationLegality::isFirstOrderRecurrence(), llvm::ScalarEvolution::isLoopInvariant(), llvm::LoopVectorizationLegality::isReductionVariable(), llvm::ElementCount::isScalar(), llvm::InnerLoopVectorizer::Legal, llvm::make_range(), Operands, llvm::InnerLoopVectorizer::OrigLoop, llvm::InnerLoopVectorizer::PSE, SI, and llvm::InnerLoopVectorizer::VF.

The documentation for this class was generated from the following files:

lib/Transforms/Vectorize/VPRecipeBuilder.h
lib/Transforms/Vectorize/LoopVectorize.cpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ VPRecipeBuilder()

Member Function Documentation

◆ createBlockInMask()

◆ createEdgeMask()

◆ createReplicateRegion()

◆ fixHeaderPhis()

◆ getRecipe()

◆ handleReplication()

◆ recordRecipeOf()

◆ setRecipe()

◆ tryToCreateWidenRecipe()