slpvectorizer::BoUpSLP::ShuffleCostEstimator Class Reference

Merges shuffle masks and emits final shuffle instruction, if required. More...

Inheritance diagram for slpvectorizer::BoUpSLP::ShuffleCostEstimator:

Public Member Functions
	ShuffleCostEstimator (Type *ScalarTy, TargetTransformInfo &TTI, ArrayRef< Value * > VectorizedVals, BoUpSLP &R, SmallPtrSetImpl< Value * > &CheckedExtracts)
Value *	adjustExtracts (const TreeEntry *E, MutableArrayRef< int > Mask, ArrayRef< std::optional< TTI::ShuffleKind > > ShuffleKinds, unsigned NumParts, bool &UseVecBaseAsInput)
std::optional< InstructionCost >	needToDelay (const TreeEntry *, ArrayRef< SmallVector< const TreeEntry * > >) const
	Checks if the specified entry E needs to be delayed because of its dependency nodes.
void	resetForSameNode ()
	Reset the builder to handle perfect diamond match.
void	add (const TreeEntry &E1, const TreeEntry &E2, ArrayRef< int > Mask)
void	add (const TreeEntry &E1, ArrayRef< int > Mask)
void	add (Value *V1, Value *V2, ArrayRef< int > Mask)
	Adds 2 input vectors and the mask for their shuffling.
void	add (Value *V1, ArrayRef< int > Mask, bool ForExtracts=false)
	Adds another one input vector and the mask for the shuffling.
Value *	gather (ArrayRef< Value * > VL, unsigned MaskVF=0, Value *Root=nullptr)
InstructionCost	createFreeze (InstructionCost Cost)
InstructionCost	finalize (ArrayRef< int > ExtMask, ArrayRef< std::pair< const TreeEntry *, unsigned > > SubVectors, ArrayRef< int > SubVectorsMask, unsigned VF=0, function_ref< void(Value *&, SmallVectorImpl< int > &, function_ref< Value *(Value *, Value *, ArrayRef< int >)>)> Action={})
	Finalize emission of the shuffles.
	~ShuffleCostEstimator ()

Additional Inherited Members
Protected Member Functions inherited from llvm::slpvectorizer::BaseShuffleAnalysis
	BaseShuffleAnalysis (Type *ScalarTy)
unsigned	getVF (Value *V) const
	V is expected to be a vectorized value.
Static Protected Member Functions inherited from llvm::slpvectorizer::BaseShuffleAnalysis
static bool	isIdentityMask (ArrayRef< int > Mask, const FixedVectorType *VecTy, bool IsStrict)
	Checks if the mask is an identity mask.
static void	combineMasks (unsigned LocalVF, SmallVectorImpl< int > &Mask, ArrayRef< int > ExtMask)
	Tries to combine 2 different masks into single one.
static bool	peekThroughShuffles (Value *&V, SmallVectorImpl< int > &Mask, bool SinglePermute)
	Looks through shuffles trying to reduce final number of shuffles in the code.
template<typename T, typename ShuffleBuilderTy, typename... Args>
static T	createShuffle (Value *V1, Value *V2, ArrayRef< int > Mask, ShuffleBuilderTy &Builder, Type *ScalarTy, bool ReVec, Args... Arguments)
	Smart shuffle instruction emission, walks through shuffles trees and tries to find the best matching vector for the actual shuffle instruction.
static void	transformMaskAfterShuffle (MutableArrayRef< int > CommonMask, ArrayRef< int > Mask)
	Transforms mask CommonMask per given Mask to make proper set after shuffle emission.
Protected Attributes inherited from llvm::slpvectorizer::BaseShuffleAnalysis
Type *	ScalarTy = nullptr

Detailed Description

Merges shuffle masks and emits final shuffle instruction, if required.

It supports shuffling of 2 input vectors. It implements lazy shuffles emission, when the actual shuffle instruction is generated only if this is actually required. Otherwise, the shuffle instruction emission is delayed till the end of the process, to reduce the number of emitted instructions and further analysis/transformations.

Definition at line 14684 of file SLPVectorizer.cpp.

Constructor & Destructor Documentation

ShuffleCostEstimator()

slpvectorizer::BoUpSLP::ShuffleCostEstimator::ShuffleCostEstimator ( Type * ScalarTy, TargetTransformInfo & TTI, ArrayRef< Value * > VectorizedVals, BoUpSLP & R, SmallPtrSetImpl< Value * > & CheckedExtracts )

inline

Definition at line 15234 of file SLPVectorizer.cpp.

References llvm::slpvectorizer::BaseShuffleAnalysis::BaseShuffleAnalysis(), slpvectorizer::BoUpSLP::BoUpSLP(), slpvectorizer::BoUpSLP::getCostKind(), and llvm::slpvectorizer::BaseShuffleAnalysis::ScalarTy.

~ShuffleCostEstimator()

slpvectorizer::BoUpSLP::ShuffleCostEstimator::~ShuffleCostEstimator ( )

inline

Definition at line 15638 of file SLPVectorizer.cpp.

References assert().

Member Function Documentation

add() [1/4]

void slpvectorizer::BoUpSLP::ShuffleCostEstimator::add ( const TreeEntry & E1, ArrayRef< int > Mask )

inline

Definition at line 15404 of file SLPVectorizer.cpp.

References assert(), llvm::find_if(), llvm::slpvectorizer::getNumberOfParts(), llvm::slpvectorizer::getPartNumElems(), llvm::slpvectorizer::getWidenedType(), llvm::not_equal_to(), llvm::PoisonMaskElem, llvm::slpvectorizer::BaseShuffleAnalysis::ScalarTy, and SLPReVec.

add() [2/4]

void slpvectorizer::BoUpSLP::ShuffleCostEstimator::add ( const TreeEntry & E1, const TreeEntry & E2, ArrayRef< int > Mask )

inline

Definition at line 15379 of file SLPVectorizer.cpp.

References add(), llvm::all_of(), assert(), llvm::find_if(), llvm::slpvectorizer::getNumberOfParts(), llvm::slpvectorizer::getPartNumElems(), llvm::slpvectorizer::getWidenedType(), llvm::not_equal_to(), llvm::PoisonMaskElem, llvm::slpvectorizer::BaseShuffleAnalysis::ScalarTy, and SLPReVec.

Referenced by add().

add() [3/4]

void slpvectorizer::BoUpSLP::ShuffleCostEstimator::add ( Value * V1, ArrayRef< int > Mask, bool ForExtracts = false )

inline

Adds another one input vector and the mask for the shuffling.

Definition at line 15440 of file SLPVectorizer.cpp.

References llvm::all_of(), assert(), llvm::cast(), llvm::enumerate(), llvm::slpvectorizer::BaseShuffleAnalysis::getVF(), llvm::isa(), P, llvm::PoisonMaskElem, llvm::slpvectorizer::BaseShuffleAnalysis::transformMaskAfterShuffle(), and llvm::V1.

add() [4/4]

void slpvectorizer::BoUpSLP::ShuffleCostEstimator::add ( Value * V1, Value * V2, ArrayRef< int > Mask )

inline

Adds 2 input vectors and the mask for their shuffling.

Definition at line 15423 of file SLPVectorizer.cpp.

References llvm::all_of(), assert(), llvm::cast(), llvm::enumerate(), P, llvm::PoisonMaskElem, and llvm::V1.

adjustExtracts()

Value * slpvectorizer::BoUpSLP::ShuffleCostEstimator::adjustExtracts ( const TreeEntry * E, MutableArrayRef< int > Mask, ArrayRef< std::optional< TTI::ShuffleKind > > ShuffleKinds, unsigned NumParts, bool & UseVecBaseAsInput )

inline

Definition at line 15240 of file SLPVectorizer.cpp.

References llvm::any_of(), llvm::ArrayRef(), and llvm::slpvectorizer::reorderScalars().

createFreeze()

InstructionCost slpvectorizer::BoUpSLP::ShuffleCostEstimator::createFreeze ( InstructionCost Cost )

inline

Definition at line 15530 of file SLPVectorizer.cpp.

finalize()

InstructionCost slpvectorizer::BoUpSLP::ShuffleCostEstimator::finalize ( ArrayRef< int > ExtMask, ArrayRef< std::pair< const TreeEntry *, unsigned > > SubVectors, ArrayRef< int > SubVectorsMask, unsigned VF = 0, function_ref< void(Value *&, SmallVectorImpl< int > &, function_ref< Value *(Value *, Value *, ArrayRef< int >)>)> Action = {} )

inline

Finalize emission of the shuffles.

Definition at line 15532 of file SLPVectorizer.cpp.

gather()

Value * slpvectorizer::BoUpSLP::ShuffleCostEstimator::gather ( ArrayRef< Value * > VL, unsigned MaskVF = 0, Value * Root = nullptr )

inline

Definition at line 15489 of file SLPVectorizer.cpp.

References assert(), llvm::ArrayRef< T >::begin(), llvm::cast(), llvm::dyn_cast(), llvm::ArrayRef< T >::end(), llvm::ArrayRef< T >::front(), llvm::ConstantVector::get(), llvm::PoisonValue::get(), llvm::UndefValue::get(), llvm::ElementCount::getFixed(), llvm::Constant::getNullValue(), llvm::Type::getScalarType(), llvm::ConstantVector::getSplat(), llvm::Value::getType(), llvm::isa(), llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::slpvectorizer::replicateMask(), llvm::slpvectorizer::BaseShuffleAnalysis::ScalarTy, llvm::ArrayRef< T >::size(), SLPReVec, and llvm::ArrayRef< T >::take_front().

needToDelay()

std::optional< InstructionCost > slpvectorizer::BoUpSLP::ShuffleCostEstimator::needToDelay ( const TreeEntry * , ArrayRef< SmallVector< const TreeEntry * > >  ) const

inline

Checks if the specified entry E needs to be delayed because of its dependency nodes.

Definition at line 15365 of file SLPVectorizer.cpp.

resetForSameNode()

void slpvectorizer::BoUpSLP::ShuffleCostEstimator::resetForSameNode ( )

inline

Reset the builder to handle perfect diamond match.

Definition at line 15371 of file SLPVectorizer.cpp.

---

The documentation for this class was generated from the following file:

• lib/Transforms/Vectorize/SLPVectorizer.cpp