VPlan models a candidate for vectorization, encoding various decisions take to produce efficient output IR, including which branches, basic-blocks and output IR instructions to generate, and their cost. More...

#include "Transforms/Vectorize/VPlan.h"

Public Member Functions
	VPlan (VPBlockBase *Entry=nullptr)

	~VPlan ()

void	prepareToExecute (Value TripCount, Value VectorTripCount, Value *CanonicalIVStartValue, VPTransformState &State)
	Prepare the plan for execution, setting up the required live-in values. More...

void	execute (struct VPTransformState *State)
	Generate the IR code for this VPlan. More...

VPBlockBase *	getEntry ()

const VPBlockBase *	getEntry () const

VPBlockBase *	setEntry (VPBlockBase *Block)

VPValue *	getOrCreateTripCount ()
	The trip count of the original loop. More...

VPValue *	getOrCreateBackedgeTakenCount ()
	The backedge taken count of the original loop. More...

VPValue &	getVectorTripCount ()
	The vector trip count. More...

void	disableValue2VPValue ()
	Mark the plan to indicate that using Value2VPValue is not safe any longer, because it may be stale. More...

void	addVF (ElementCount VF)

bool	hasVF (ElementCount VF)

const std::string &	getName () const

void	setName (const Twine &newName)

VPValue *	getOrAddExternalDef (Value *V)
	Get the existing or add a new external definition for `V`. More...

void	addVPValue (Value *V)

void	addVPValue (Value V, VPValue VPV)

VPValue *	getVPValue (Value *V, bool OverrideAllowed=false)
	Returns the VPValue for `V`. More...

VPValue *	getOrAddVPValue (Value *V, bool OverrideAllowed=false)
	Gets the VPValue or adds a new one (if none exists yet) for `V`. More...

void	removeVPValueFor (Value *V)

VPLoopInfo &	getVPLoopInfo ()
	Return the VPLoopInfo analysis for this VPlan. More...

const VPLoopInfo &	getVPLoopInfo () const

void	print (raw_ostream &O) const
	Print this VPlan to `O`. More...

void	printDOT (raw_ostream &O) const
	Print this VPlan in DOT format to `O`. More...

LLVM_DUMP_METHOD void	dump () const
	Dump the plan to stderr (for debugging). More...

iterator_range< mapped_iterator< Use , std::function< VPValue (Value *)> > >	mapToVPValues (User::op_range Operands)
	Returns a range mapping the values the range `Operands` to their corresponding VPValues. More...

bool	isUniformAfterVectorization (VPValue *VPV) const
	Returns true if `VPV` is uniform after vectorization. More...

VPRegionBlock *	getVectorLoopRegion ()
	Returns the VPRegionBlock of the vector loop. More...

const VPRegionBlock *	getVectorLoopRegion () const

VPCanonicalIVPHIRecipe *	getCanonicalIV ()
	Returns the canonical induction recipe of the vector loop. More...

void	addLiveOut (PHINode PN, VPValue V)

void	clearLiveOuts ()

void	removeLiveOut (PHINode *PN)

const DenseMap< PHINode , VPLiveOut > &	getLiveOuts () const

Friends
class	VPlanPrinter

class	VPSlotTracker

Detailed Description

VPlan models a candidate for vectorization, encoding various decisions take to produce efficient output IR, including which branches, basic-blocks and output IR instructions to generate, and their cost.

VPlan holds a Hierarchical-CFG of VPBasicBlocks and VPRegionBlocks rooted at an Entry VPBlock.

Definition at line 2472 of file VPlan.h.

Constructor & Destructor Documentation

◆ VPlan()

llvm::VPlan::VPlan ( VPBlockBase * Entry = nullptr )

inline

Definition at line 2519 of file VPlan.h.

◆ ~VPlan()

llvm::VPlan::~VPlan ( )

inline

Definition at line 2524 of file VPlan.h.

References clearLiveOuts().

Member Function Documentation

◆ addLiveOut()

void VPlan::addLiveOut	(	PHINode *	PN,
		VPValue *	V
	)

Definition at line 1099 of file VPlan.cpp.

References assert().

Referenced by addUsersInExitBlock().

◆ addVF()

void llvm::VPlan::addVF ( ElementCount VF )

inline

Definition at line 2581 of file VPlan.h.

References llvm::SetVector< T, SmallVector< T, N >, SmallDenseSet< T, N > >::insert().

◆ addVPValue() [1/2]

void llvm::VPlan::addVPValue ( Value * V )

inline

Definition at line 2597 of file VPlan.h.

References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count().

Referenced by getOrAddVPValue().

◆ addVPValue() [2/2]

void llvm::VPlan::addVPValue	(	Value *	V,
		VPValue *	VPV
	)

inline

Definition at line 2607 of file VPlan.h.

References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count().

◆ clearLiveOuts()

void llvm::VPlan::clearLiveOuts ( )

inline

Definition at line 2697 of file VPlan.h.

Referenced by llvm::InnerLoopVectorizer::fixVectorizedLoop(), and ~VPlan().

◆ disableValue2VPValue()

void llvm::VPlan::disableValue2VPValue ( )

inline

Mark the plan to indicate that using Value2VPValue is not safe any longer, because it may be stale.

Definition at line 2579 of file VPlan.h.

◆ dump()

LLVM_DUMP_METHOD void VPlan::dump ( ) const

Dump the plan to stderr (for debugging).

Definition at line 1096 of file VPlan.cpp.

References llvm::dbgs(), and print().

◆ execute()

void VPlan::execute ( struct VPTransformState * State )

Generate the IR code for this VPlan.

Generate the code inside the preheader and body of the vectorized loop.

Assumes a single pre-header basic-block was created for this. Introduce additional basic-blocks as needed, and fill them all.

Definition at line 956 of file VPlan.cpp.

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ getCanonicalIV()

VPCanonicalIVPHIRecipe* llvm::VPlan::getCanonicalIV ( )

inline

Returns the canonical induction recipe of the vector loop.

Definition at line 2686 of file VPlan.h.

References llvm::VPBasicBlock::begin(), llvm::VPBasicBlock::empty(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPBlockBase::getSingleSuccessor(), and getVectorLoopRegion().

Referenced by llvm::VPlanTransforms::optimizeInductions(), and llvm::VPlanTransforms::removeRedundantCanonicalIVs().

◆ getEntry() [1/2]

VPBlockBase* llvm::VPlan::getEntry ( )

inline

Definition at line 2551 of file VPlan.h.

Referenced by llvm::InnerLoopVectorizer::collectPoisonGeneratingRecipes(), llvm::VPlanPrinter::dump(), llvm::vputils::getOrCreateVPValueForSCEVExpr(), getVectorLoopRegion(), llvm::VPlanTransforms::mergeReplicateRegions(), llvm::VPlanPredicator::predicate(), llvm::VPlanTransforms::removeRedundantExpandSCEVRecipes(), llvm::VPBlockBase::setPlan(), llvm::VPlanTransforms::sinkScalarOperands(), llvm::VPlanVerifier::verifyPlanIsValid(), and llvm::VPlanPredicator::VPlanPredicator().

◆ getEntry() [2/2]

const VPBlockBase* llvm::VPlan::getEntry ( ) const

inline

Definition at line 2552 of file VPlan.h.

◆ getLiveOuts()

const DenseMap<PHINode *, VPLiveOut *>& llvm::VPlan::getLiveOuts ( ) const

inline

Definition at line 2708 of file VPlan.h.

◆ getName()

const std::string& llvm::VPlan::getName ( ) const

inline

Definition at line 2585 of file VPlan.h.

Referenced by llvm::VPlanPrinter::dump().

◆ getOrAddExternalDef()

VPValue* llvm::VPlan::getOrAddExternalDef ( Value * V )

inline

Get the existing or add a new external definition for V.

Definition at line 2590 of file VPlan.h.

References I, and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::insert().

Referenced by llvm::vputils::getOrCreateVPValueForSCEVExpr(), and llvm::LoopVectorizePass::processLoop().

◆ getOrAddVPValue()

VPValue* llvm::VPlan::getOrAddVPValue	(	Value *	V,
		bool	OverrideAllowed = `false`
	)

inline

Gets the VPValue or adds a new one (if none exists yet) for V.

OverrideAllowed can be used to disable checking whether it is safe to query VPValues using IR Values.

Definition at line 2627 of file VPlan.h.

References addVPValue(), assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count(), and getVPValue().

Referenced by addCanonicalIVRecipes(), addUsersInExitBlock(), and mapToVPValues().

◆ getOrCreateBackedgeTakenCount()

VPValue* llvm::VPlan::getOrCreateBackedgeTakenCount ( )

inline

The backedge taken count of the original loop.

Definition at line 2568 of file VPlan.h.

◆ getOrCreateTripCount()

VPValue* llvm::VPlan::getOrCreateTripCount ( )

inline

The trip count of the original loop.

Definition at line 2561 of file VPlan.h.

◆ getVectorLoopRegion() [1/2]

VPRegionBlock* llvm::VPlan::getVectorLoopRegion ( )

inline

Returns the VPRegionBlock of the vector loop.

Definition at line 2674 of file VPlan.h.

References getEntry(), and llvm::RISCVFenceField::R.

Referenced by addCanonicalIVRecipes(), llvm::LoopVectorizationPlanner::executePlan(), llvm::InnerLoopVectorizer::fixCrossIterationPHIs(), llvm::InnerLoopVectorizer::fixVectorizedLoop(), getCanonicalIV(), llvm::VPlanTransforms::optimizeInductions(), llvm::LoopVectorizePass::processLoop(), llvm::VPlanTransforms::removeDeadRecipes(), llvm::VPlanTransforms::removeRedundantCanonicalIVs(), llvm::VPlanTransforms::removeRedundantInductionCasts(), and llvm::VPInterleavedAccessInfo::VPInterleavedAccessInfo().

◆ getVectorLoopRegion() [2/2]

const VPRegionBlock* llvm::VPlan::getVectorLoopRegion ( ) const

inline

Definition at line 2679 of file VPlan.h.

References getEntry(), and llvm::RISCVFenceField::R.

◆ getVectorTripCount()

VPValue& llvm::VPlan::getVectorTripCount ( )

inline

The vector trip count.

Definition at line 2575 of file VPlan.h.

Referenced by addCanonicalIVRecipes().

◆ getVPLoopInfo() [1/2]

VPLoopInfo& llvm::VPlan::getVPLoopInfo ( )

inline

Return the VPLoopInfo analysis for this VPlan.

Definition at line 2643 of file VPlan.h.

Referenced by llvm::VPlanHCFGBuilder::buildHierarchicalCFG().

◆ getVPLoopInfo() [2/2]

const VPLoopInfo& llvm::VPlan::getVPLoopInfo ( ) const

inline

Definition at line 2644 of file VPlan.h.

◆ getVPValue()

VPValue* llvm::VPlan::getVPValue	(	Value *	V,
		bool	OverrideAllowed = `false`
	)

inline

Returns the VPValue for V.

OverrideAllowed can be used to disable checking whether it is safe to query VPValues using IR Values.

Definition at line 2616 of file VPlan.h.

References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count().

Referenced by llvm::InnerLoopVectorizer::fixNonInductionPHIs(), getOrAddVPValue(), and llvm::InnerLoopVectorizer::truncateToMinimalBitwidths().

◆ hasVF()

bool llvm::VPlan::hasVF ( ElementCount VF )

inline

Definition at line 2583 of file VPlan.h.

References llvm::SetVector< T, SmallVector< T, N >, SmallDenseSet< T, N > >::count().

◆ isUniformAfterVectorization()

bool llvm::VPlan::isUniformAfterVectorization ( VPValue * VPV ) const

inline

Returns true if VPV is uniform after vectorization.

Definition at line 2668 of file VPlan.h.

References llvm::VPValue::getDef().

Referenced by llvm::VPLiveOut::fixPhi().

◆ mapToVPValues()

iterator_range<mapped_iterator<Use *, std::function<VPValue *(Value *)> > > llvm::VPlan::mapToVPValues ( User::op_range Operands )

inline

Returns a range mapping the values the range Operands to their corresponding VPValues.

Definition at line 2660 of file VPlan.h.

References function, getOrAddVPValue(), llvm::map_range(), and Operands.

◆ prepareToExecute()

void VPlan::prepareToExecute	(	Value *	TripCount,
		Value *	VectorTripCount,
		Value *	CanonicalIVStartValue,
		VPTransformState &	State
	)

Prepare the plan for execution, setting up the required live-in values.

Definition at line 906 of file VPlan.cpp.

References llvm::all_of(), assert(), llvm::VPTransformState::Builder, llvm::VPTransformState::CFG, llvm::IRBuilderBase::CreateSub(), llvm::IRBuilderBase::CreateVectorSplat(), llvm::ConstantInt::get(), llvm::BasicBlock::getTerminator(), llvm::Value::getType(), llvm::ElementCount::isScalar(), IV, llvm::VPTransformState::CFGState::PrevBB, llvm::VPTransformState::set(), llvm::VPTransformState::UF, and llvm::VPTransformState::VF.

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ print()

LLVM_DUMP_METHOD void VPlan::print ( raw_ostream & O ) const

Print this VPlan to O.

Definition at line 1054 of file VPlan.cpp.

References llvm::depth_first(), and llvm::RISCVFenceField::O.

Referenced by llvm::operator<<().

◆ printDOT()

LLVM_DUMP_METHOD void VPlan::printDOT ( raw_ostream & O ) const

Print this VPlan in DOT format to O.

Definition at line 1090 of file VPlan.cpp.

References llvm::RISCVFenceField::O, and Printer.

◆ removeLiveOut()

void llvm::VPlan::removeLiveOut ( PHINode * PN )

inline

Definition at line 2703 of file VPlan.h.

References llvm::DenseMapBase< DenseMap< KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >, KeyT, ValueT, DenseMapInfo< KeyT >, llvm::detail::DenseMapPair< KeyT, ValueT > >::erase().

Referenced by llvm::InnerLoopVectorizer::fixFirstOrderRecurrence(), llvm::InnerLoopVectorizer::fixReduction(), and llvm::InnerLoopVectorizer::fixupIVUsers().

◆ removeVPValueFor()

void llvm::VPlan::removeVPValueFor ( Value * V )

inline

Definition at line 2636 of file VPlan.h.

References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::erase().

◆ setEntry()

VPBlockBase* llvm::VPlan::setEntry ( VPBlockBase * Block )

inline

Definition at line 2554 of file VPlan.h.

Referenced by llvm::VPlanHCFGBuilder::buildHierarchicalCFG().

◆ setName()

void llvm::VPlan::setName ( const Twine & newName )

inline

Definition at line 2587 of file VPlan.h.

References llvm::Twine::str().

Referenced by llvm::VPlanHCFGBuilder::buildHierarchicalCFG().

Friends And Related Function Documentation

◆ VPlanPrinter

friend class VPlanPrinter

friend

Definition at line 2473 of file VPlan.h.

◆ VPSlotTracker

friend class VPSlotTracker

friend

Definition at line 2474 of file VPlan.h.

The documentation for this class was generated from the following files:

lib/Transforms/Vectorize/VPlan.h
lib/Transforms/Vectorize/VPlan.cpp

Public Member Functions

Friends

Detailed Description

Constructor & Destructor Documentation

◆ VPlan()

◆ ~VPlan()

Member Function Documentation

◆ addLiveOut()

◆ addVF()

◆ addVPValue() [1/2]

◆ addVPValue() [2/2]

◆ clearLiveOuts()

◆ disableValue2VPValue()

◆ dump()

◆ execute()

◆ getCanonicalIV()

◆ getEntry() [1/2]

◆ getEntry() [2/2]

◆ getLiveOuts()

◆ getName()

◆ getOrAddExternalDef()

◆ getOrAddVPValue()

◆ getOrCreateBackedgeTakenCount()

◆ getOrCreateTripCount()

◆ getVectorLoopRegion() [1/2]

◆ getVectorLoopRegion() [2/2]

◆ getVectorTripCount()

◆ getVPLoopInfo() [1/2]

◆ getVPLoopInfo() [2/2]

◆ getVPValue()

◆ hasVF()

◆ isUniformAfterVectorization()

◆ mapToVPValues()

◆ prepareToExecute()

◆ print()

◆ printDOT()

◆ removeLiveOut()

◆ removeVPValueFor()

◆ setEntry()

◆ setName()

Friends And Related Function Documentation

◆ VPlanPrinter

◆ VPSlotTracker