LLVM
15.0.0git
|
VPlan models a candidate for vectorization, encoding various decisions take to produce efficient output IR, including which branches, basic-blocks and output IR instructions to generate, and their cost. More...
#include "Transforms/Vectorize/VPlan.h"
Public Member Functions | |
VPlan (VPBlockBase *Entry=nullptr) | |
~VPlan () | |
void | prepareToExecute (Value *TripCount, Value *VectorTripCount, Value *CanonicalIVStartValue, VPTransformState &State) |
Prepare the plan for execution, setting up the required live-in values. More... | |
void | execute (struct VPTransformState *State) |
Generate the IR code for this VPlan. More... | |
VPBlockBase * | getEntry () |
const VPBlockBase * | getEntry () const |
VPBlockBase * | setEntry (VPBlockBase *Block) |
VPValue * | getOrCreateTripCount () |
The trip count of the original loop. More... | |
VPValue * | getOrCreateBackedgeTakenCount () |
The backedge taken count of the original loop. More... | |
VPValue & | getVectorTripCount () |
The vector trip count. More... | |
void | disableValue2VPValue () |
Mark the plan to indicate that using Value2VPValue is not safe any longer, because it may be stale. More... | |
void | addVF (ElementCount VF) |
bool | hasVF (ElementCount VF) |
const std::string & | getName () const |
void | setName (const Twine &newName) |
VPValue * | getOrAddExternalDef (Value *V) |
Get the existing or add a new external definition for V . More... | |
void | addVPValue (Value *V) |
void | addVPValue (Value *V, VPValue *VPV) |
VPValue * | getVPValue (Value *V, bool OverrideAllowed=false) |
Returns the VPValue for V . More... | |
VPValue * | getOrAddVPValue (Value *V, bool OverrideAllowed=false) |
Gets the VPValue or adds a new one (if none exists yet) for V . More... | |
void | removeVPValueFor (Value *V) |
VPLoopInfo & | getVPLoopInfo () |
Return the VPLoopInfo analysis for this VPlan. More... | |
const VPLoopInfo & | getVPLoopInfo () const |
void | print (raw_ostream &O) const |
Print this VPlan to O . More... | |
void | printDOT (raw_ostream &O) const |
Print this VPlan in DOT format to O . More... | |
LLVM_DUMP_METHOD void | dump () const |
Dump the plan to stderr (for debugging). More... | |
iterator_range< mapped_iterator< Use *, std::function< VPValue *(Value *)> > > | mapToVPValues (User::op_range Operands) |
Returns a range mapping the values the range Operands to their corresponding VPValues. More... | |
bool | isUniformAfterVectorization (VPValue *VPV) const |
Returns true if VPV is uniform after vectorization. More... | |
VPRegionBlock * | getVectorLoopRegion () |
Returns the VPRegionBlock of the vector loop. More... | |
const VPRegionBlock * | getVectorLoopRegion () const |
VPCanonicalIVPHIRecipe * | getCanonicalIV () |
Returns the canonical induction recipe of the vector loop. More... | |
void | addLiveOut (PHINode *PN, VPValue *V) |
void | clearLiveOuts () |
void | removeLiveOut (PHINode *PN) |
const DenseMap< PHINode *, VPLiveOut * > & | getLiveOuts () const |
Friends | |
class | VPlanPrinter |
class | VPSlotTracker |
VPlan models a candidate for vectorization, encoding various decisions take to produce efficient output IR, including which branches, basic-blocks and output IR instructions to generate, and their cost.
VPlan holds a Hierarchical-CFG of VPBasicBlocks and VPRegionBlocks rooted at an Entry VPBlock.
|
inline |
|
inline |
Definition at line 2524 of file VPlan.h.
References clearLiveOuts().
Definition at line 1099 of file VPlan.cpp.
References assert().
Referenced by addUsersInExitBlock().
|
inline |
Definition at line 2581 of file VPlan.h.
References llvm::SetVector< T, SmallVector< T, N >, SmallDenseSet< T, N > >::insert().
|
inline |
Definition at line 2597 of file VPlan.h.
References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count().
Referenced by getOrAddVPValue().
Definition at line 2607 of file VPlan.h.
References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count().
|
inline |
Definition at line 2697 of file VPlan.h.
Referenced by llvm::InnerLoopVectorizer::fixVectorizedLoop(), and ~VPlan().
|
inline |
LLVM_DUMP_METHOD void VPlan::dump | ( | ) | const |
Dump the plan to stderr (for debugging).
Definition at line 1096 of file VPlan.cpp.
References llvm::dbgs(), and print().
void VPlan::execute | ( | struct VPTransformState * | State | ) |
Generate the IR code for this VPlan.
Generate the code inside the preheader and body of the vectorized loop.
Assumes a single pre-header basic-block was created for this. Introduce additional basic-blocks as needed, and fill them all.
Definition at line 956 of file VPlan.cpp.
Referenced by llvm::LoopVectorizationPlanner::executePlan().
|
inline |
Returns the canonical induction recipe of the vector loop.
Definition at line 2686 of file VPlan.h.
References llvm::VPBasicBlock::begin(), llvm::VPBasicBlock::empty(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPBlockBase::getSingleSuccessor(), and getVectorLoopRegion().
Referenced by llvm::VPlanTransforms::optimizeInductions(), and llvm::VPlanTransforms::removeRedundantCanonicalIVs().
|
inline |
Definition at line 2551 of file VPlan.h.
Referenced by llvm::InnerLoopVectorizer::collectPoisonGeneratingRecipes(), llvm::VPlanPrinter::dump(), llvm::vputils::getOrCreateVPValueForSCEVExpr(), getVectorLoopRegion(), llvm::VPlanTransforms::mergeReplicateRegions(), llvm::VPlanPredicator::predicate(), llvm::VPlanTransforms::removeRedundantExpandSCEVRecipes(), llvm::VPBlockBase::setPlan(), llvm::VPlanTransforms::sinkScalarOperands(), llvm::VPlanVerifier::verifyPlanIsValid(), and llvm::VPlanPredicator::VPlanPredicator().
|
inline |
|
inline |
Definition at line 2585 of file VPlan.h.
Referenced by llvm::VPlanPrinter::dump().
Get the existing or add a new external definition for V
.
Definition at line 2590 of file VPlan.h.
References I, and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::insert().
Referenced by llvm::vputils::getOrCreateVPValueForSCEVExpr(), and llvm::LoopVectorizePass::processLoop().
Gets the VPValue or adds a new one (if none exists yet) for V
.
OverrideAllowed
can be used to disable checking whether it is safe to query VPValues using IR Values.
Definition at line 2627 of file VPlan.h.
References addVPValue(), assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count(), and getVPValue().
Referenced by addCanonicalIVRecipes(), addUsersInExitBlock(), and mapToVPValues().
|
inline |
|
inline |
|
inline |
Returns the VPRegionBlock of the vector loop.
Definition at line 2674 of file VPlan.h.
References getEntry(), and llvm::RISCVFenceField::R.
Referenced by addCanonicalIVRecipes(), llvm::LoopVectorizationPlanner::executePlan(), llvm::InnerLoopVectorizer::fixCrossIterationPHIs(), llvm::InnerLoopVectorizer::fixVectorizedLoop(), getCanonicalIV(), llvm::VPlanTransforms::optimizeInductions(), llvm::LoopVectorizePass::processLoop(), llvm::VPlanTransforms::removeDeadRecipes(), llvm::VPlanTransforms::removeRedundantCanonicalIVs(), llvm::VPlanTransforms::removeRedundantInductionCasts(), and llvm::VPInterleavedAccessInfo::VPInterleavedAccessInfo().
|
inline |
Definition at line 2679 of file VPlan.h.
References getEntry(), and llvm::RISCVFenceField::R.
|
inline |
The vector trip count.
Definition at line 2575 of file VPlan.h.
Referenced by addCanonicalIVRecipes().
|
inline |
Return the VPLoopInfo analysis for this VPlan.
Definition at line 2643 of file VPlan.h.
Referenced by llvm::VPlanHCFGBuilder::buildHierarchicalCFG().
|
inline |
Returns the VPValue for V
.
OverrideAllowed
can be used to disable checking whether it is safe to query VPValues using IR Values.
Definition at line 2616 of file VPlan.h.
References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count().
Referenced by llvm::InnerLoopVectorizer::fixNonInductionPHIs(), getOrAddVPValue(), and llvm::InnerLoopVectorizer::truncateToMinimalBitwidths().
|
inline |
Definition at line 2583 of file VPlan.h.
References llvm::SetVector< T, SmallVector< T, N >, SmallDenseSet< T, N > >::count().
|
inline |
Returns true if VPV
is uniform after vectorization.
Definition at line 2668 of file VPlan.h.
References llvm::VPValue::getDef().
Referenced by llvm::VPLiveOut::fixPhi().
|
inline |
Returns a range mapping the values the range Operands
to their corresponding VPValues.
Definition at line 2660 of file VPlan.h.
References function, getOrAddVPValue(), llvm::map_range(), and Operands.
void VPlan::prepareToExecute | ( | Value * | TripCount, |
Value * | VectorTripCount, | ||
Value * | CanonicalIVStartValue, | ||
VPTransformState & | State | ||
) |
Prepare the plan for execution, setting up the required live-in values.
Definition at line 906 of file VPlan.cpp.
References llvm::all_of(), assert(), llvm::VPTransformState::Builder, llvm::VPTransformState::CFG, llvm::IRBuilderBase::CreateSub(), llvm::IRBuilderBase::CreateVectorSplat(), llvm::ConstantInt::get(), llvm::BasicBlock::getTerminator(), llvm::Value::getType(), llvm::ElementCount::isScalar(), IV, llvm::VPTransformState::CFGState::PrevBB, llvm::VPTransformState::set(), llvm::VPTransformState::UF, and llvm::VPTransformState::VF.
Referenced by llvm::LoopVectorizationPlanner::executePlan().
LLVM_DUMP_METHOD void VPlan::print | ( | raw_ostream & | O | ) | const |
Print this VPlan to O
.
Definition at line 1054 of file VPlan.cpp.
References llvm::depth_first(), and llvm::RISCVFenceField::O.
Referenced by llvm::operator<<().
LLVM_DUMP_METHOD void VPlan::printDOT | ( | raw_ostream & | O | ) | const |
|
inline |
Definition at line 2703 of file VPlan.h.
Referenced by llvm::InnerLoopVectorizer::fixFirstOrderRecurrence(), llvm::InnerLoopVectorizer::fixReduction(), and llvm::InnerLoopVectorizer::fixupIVUsers().
|
inline |
Definition at line 2636 of file VPlan.h.
References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::erase().
|
inline |
Definition at line 2554 of file VPlan.h.
Referenced by llvm::VPlanHCFGBuilder::buildHierarchicalCFG().
Definition at line 2587 of file VPlan.h.
References llvm::Twine::str().
Referenced by llvm::VPlanHCFGBuilder::buildHierarchicalCFG().
|
friend |
|
friend |