LLVM
15.0.0git
|
Helper class to create VPRecipies from IR instructions. More...
#include "Transforms/Vectorize/VPRecipeBuilder.h"
Public Member Functions | |
VPRecipeBuilder (Loop *OrigLoop, const TargetLibraryInfo *TLI, LoopVectorizationLegality *Legal, LoopVectorizationCostModel &CM, PredicatedScalarEvolution &PSE, VPBuilder &Builder) | |
VPRecipeOrVPValueTy | tryToCreateWidenRecipe (Instruction *Instr, ArrayRef< VPValue * > Operands, VFRange &Range, VPlanPtr &Plan) |
Check if an existing VPValue can be used for Instr or a recipe can be create for I withing the given VF Range . More... | |
void | setRecipe (Instruction *I, VPRecipeBase *R) |
Set the recipe created for given ingredient. More... | |
VPValue * | createBlockInMask (BasicBlock *BB, VPlanPtr &Plan) |
A helper function that computes the predicate of the block BB, assuming that the header block of the loop is set to True. More... | |
VPValue * | createEdgeMask (BasicBlock *Src, BasicBlock *Dst, VPlanPtr &Plan) |
A helper function that computes the predicate of the edge between SRC and DST. More... | |
void | recordRecipeOf (Instruction *I) |
Mark given ingredient for recording its recipe once one is created for it. More... | |
VPRecipeBase * | getRecipe (Instruction *I) |
Return the recipe created for given ingredient. More... | |
VPRegionBlock * | createReplicateRegion (Instruction *I, VPRecipeBase *PredRecipe, VPlanPtr &Plan) |
Create a replicating region for instruction I that requires predication. More... | |
VPBasicBlock * | handleReplication (Instruction *I, VFRange &Range, VPBasicBlock *VPBB, VPlanPtr &Plan) |
Build a VPReplicationRecipe for I and enclose it within a Region if it is predicated. More... | |
void | fixHeaderPhis () |
Add the incoming values from the backedge to reduction & first-order recurrence cross-iteration phis. More... | |
Helper class to create VPRecipies from IR instructions.
Definition at line 27 of file VPRecipeBuilder.h.
|
inline |
Definition at line 109 of file VPRecipeBuilder.h.
References Builder, and llvm::IRSimilarity::Legal.
VPValue * VPRecipeBuilder::createBlockInMask | ( | BasicBlock * | BB, |
VPlanPtr & | Plan | ||
) |
A helper function that computes the predicate of the block BB, assuming that the header block of the loop is set to True.
It returns the entry mask for the block BB.
Definition at line 8054 of file LoopVectorize.cpp.
References llvm::VPInstruction::ActiveLaneMask, assert(), BB, llvm::InnerLoopVectorizer::Builder, llvm::LoopBase< BlockT, LoopT >::contains(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPBasicBlock::getFirstNonPhi(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::VPInstruction::ICmpULE, llvm::VPBasicBlock::insert(), IV, llvm::InnerLoopVectorizer::OrigLoop, and llvm::predecessors().
VPValue * VPRecipeBuilder::createEdgeMask | ( | BasicBlock * | Src, |
BasicBlock * | Dst, | ||
VPlanPtr & | Plan | ||
) |
A helper function that computes the predicate of the edge between SRC and DST.
Definition at line 8009 of file LoopVectorize.cpp.
References assert(), llvm::InnerLoopVectorizer::Builder, llvm::BranchInst::getCondition(), llvm::Instruction::getDebugLoc(), llvm::ConstantInt::getFalse(), llvm::BranchInst::getSuccessor(), llvm::Value::getType(), llvm::is_contained(), llvm::BranchInst::isConditional(), llvm::LoopBase< BlockT, LoopT >::isLoopExiting(), llvm::InnerLoopVectorizer::OrigLoop, and llvm::predecessors().
VPRegionBlock * VPRecipeBuilder::createReplicateRegion | ( | Instruction * | I, |
VPRecipeBase * | PredRecipe, | ||
VPlanPtr & | Plan | ||
) |
Create a replicating region for instruction I
that requires predication.
PredRecipe
is a VPReplicateRecipe holding I
.
Definition at line 8478 of file LoopVectorize.cpp.
References assert(), llvm::Instruction::getOpcodeName(), and llvm::Instruction::getParent().
void VPRecipeBuilder::fixHeaderPhis | ( | ) |
Add the incoming values from the backedge to reduction & first-order recurrence cross-iteration phis.
Definition at line 8386 of file LoopVectorize.cpp.
References llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), llvm::VPDef::getVPSingleValue(), and llvm::InnerLoopVectorizer::OrigLoop.
|
inline |
Return the recipe created for given ingredient.
Definition at line 152 of file VPRecipeBuilder.h.
References assert(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >, KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >::count(), and I.
VPBasicBlock * VPRecipeBuilder::handleReplication | ( | Instruction * | I, |
VFRange & | Range, | ||
VPBasicBlock * | VPBB, | ||
VPlanPtr & | Plan | ||
) |
Build a VPReplicationRecipe for I
and enclose it within a Region if it is predicated.
VPBB
augmented with this new recipe if I
is not predicated, otherwise I
. Range.End may be decreased to ensure same recipe behavior from Range.Start
to Range.End
. Definition at line 8396 of file LoopVectorize.cpp.
References llvm::VPBasicBlock::appendRecipe(), assert(), llvm::VPBlockUtils::connectBlocks(), llvm::dbgs(), llvm::VPBlockUtils::disconnectBlocks(), llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), getIntrinsicID(), llvm::VPBlockBase::getSingleSuccessor(), I, llvm::VPBlockUtils::insertBlockAfter(), llvm::LinearPolySize< LeafTy >::isScalable(), LLVM_DEBUG, llvm::VFRange::Start, and llvm::InnerLoopVectorizer::VF.
|
inline |
Mark given ingredient for recording its recipe once one is created for it.
Definition at line 145 of file VPRecipeBuilder.h.
References assert(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >, KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >::count(), and I.
|
inline |
Set the recipe created for given ingredient.
This operation is a no-op for ingredients that were not marked using a nullptr entry in the map.
Definition at line 126 of file VPRecipeBuilder.h.
References assert(), llvm::DenseMapBase< DenseMap< KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >, KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >::count(), and I.
VPRecipeOrVPValueTy VPRecipeBuilder::tryToCreateWidenRecipe | ( | Instruction * | Instr, |
ArrayRef< VPValue * > | Operands, | ||
VFRange & | Range, | ||
VPlanPtr & | Plan | ||
) |
Check if an existing VPValue can be used for Instr
or a recipe can be create for I
withing the given VF Range
.
If an existing VPValue can be used or if a recipe can be created, return it. Otherwise return a VPRecipeOrVPValueTy with nullptr.
Definition at line 8512 of file LoopVectorize.cpp.
References assert(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::find(), GEP, llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::PHINode::getIncomingValueForBlock(), llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), llvm::LoopBase< BlockT, LoopT >::getLoopPreheader(), llvm::Instruction::getParent(), llvm::RecurrenceDescriptor::getRecurrenceStartValue(), llvm::LoopVectorizationLegality::getReductionVars(), llvm::PredicatedScalarEvolution::getSCEV(), llvm::PredicatedScalarEvolution::getSE(), llvm::LoopVectorizationLegality::isFirstOrderRecurrence(), llvm::ScalarEvolution::isLoopInvariant(), llvm::LoopVectorizationLegality::isReductionVariable(), llvm::ElementCount::isScalar(), llvm::InnerLoopVectorizer::Legal, llvm::make_range(), Operands, llvm::InnerLoopVectorizer::OrigLoop, llvm::InnerLoopVectorizer::PSE, SI, and llvm::InnerLoopVectorizer::VF.