LLVM 19.0.0git
Classes | Public Member Functions | List of all members
llvm::InterleavedAccessInfo Class Reference

Drive the analysis of interleaved memory accesses in the loop. More...

#include "llvm/Analysis/VectorUtils.h"

Public Member Functions

 InterleavedAccessInfo (PredicatedScalarEvolution &PSE, Loop *L, DominatorTree *DT, LoopInfo *LI, const LoopAccessInfo *LAI)
 
 ~InterleavedAccessInfo ()
 
void analyzeInterleaving (bool EnableMaskedInterleavedGroup)
 Analyze the interleaved accesses and collect them in interleave groups.
 
bool invalidateGroups ()
 Invalidate groups, e.g., in case all blocks in loop will be predicated contrary to original assumption.
 
bool isInterleaved (Instruction *Instr) const
 Check if Instr belongs to any interleave group.
 
InterleaveGroup< Instruction > * getInterleaveGroup (const Instruction *Instr) const
 Get the interleave group that Instr belongs to.
 
iterator_range< SmallPtrSetIterator< llvm::InterleaveGroup< Instruction > * > > getInterleaveGroups ()
 
bool requiresScalarEpilogue () const
 Returns true if an interleaved group that may access memory out-of-bounds requires a scalar epilogue iteration for correctness.
 
void invalidateGroupsRequiringScalarEpilogue ()
 Invalidate groups that require a scalar epilogue (due to gaps).
 
bool hasGroups () const
 Returns true if we have any interleave groups.
 

Detailed Description

Drive the analysis of interleaved memory accesses in the loop.

Use this class to analyze interleaved accesses only when we can vectorize a loop. Otherwise it's meaningless to do analysis as the vectorization on interleaved accesses is unsafe.

The analysis collects interleave groups and records the relationships between the member and the group in a map.

Definition at line 587 of file VectorUtils.h.

Constructor & Destructor Documentation

◆ InterleavedAccessInfo()

llvm::InterleavedAccessInfo::InterleavedAccessInfo ( PredicatedScalarEvolution PSE,
Loop L,
DominatorTree DT,
LoopInfo LI,
const LoopAccessInfo LAI 
)
inline

Definition at line 589 of file VectorUtils.h.

◆ ~InterleavedAccessInfo()

llvm::InterleavedAccessInfo::~InterleavedAccessInfo ( )
inline

Definition at line 594 of file VectorUtils.h.

References invalidateGroups().

Member Function Documentation

◆ analyzeInterleaving()

void InterleavedAccessInfo::analyzeInterleaving ( bool  EnableMaskedInterleavedGroup)

Analyze the interleaved accesses and collect them in interleave groups.

Substitute symbolic strides using Strides. Consider also predicated loads/stores in the analysis if EnableMaskedInterleavedGroup is true.

Definition at line 1149 of file VectorUtils.cpp.

References A, assert(), B, llvm::SetVector< T, Vector, Set, N >::contains(), llvm::SmallPtrSetImpl< PtrType >::contains(), llvm::dbgs(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::empty(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::find(), llvm::SCEVConstant::getAPInt(), llvm::InterleaveGroup< InstTy >::getFactor(), llvm::InterleaveGroup< InstTy >::getIndex(), getInterleaveGroup(), llvm::getLoadStoreAddressSpace(), llvm::getLoadStorePointerOperand(), llvm::getLoadStoreType(), llvm::InterleaveGroup< InstTy >::getMember(), llvm::ScalarEvolution::getMinusSCEV(), llvm::InterleaveGroup< InstTy >::getNumMembers(), llvm::getPtrStride(), llvm::PredicatedScalarEvolution::getSE(), llvm::APInt::getSExtValue(), llvm::LoopAccessInfo::getSymbolicStrides(), llvm::SetVector< T, Vector, Set, N >::insert(), llvm::SmallPtrSetImpl< PtrType >::insert(), llvm::InterleaveGroup< InstTy >::insertMember(), isInterleaved(), llvm::InterleaveGroup< InstTy >::isReverse(), LLVM_DEBUG, llvm::MapVector< KeyT, ValueT, MapType, VectorType >::rbegin(), llvm::SetVector< T, Vector, Set, N >::remove(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::rend(), and llvm::InterleaveGroup< InstTy >::setInsertPos().

Referenced by llvm::LoopVectorizePass::processLoop().

◆ getInterleaveGroup()

InterleaveGroup< Instruction > * llvm::InterleavedAccessInfo::getInterleaveGroup ( const Instruction Instr) const
inline

Get the interleave group that Instr belongs to.

Returns
nullptr if doesn't have such group.

Definition at line 632 of file VectorUtils.h.

Referenced by analyzeInterleaving(), and llvm::LoopVectorizationCostModel::getInterleavedAccessGroup().

◆ getInterleaveGroups()

iterator_range< SmallPtrSetIterator< llvm::InterleaveGroup< Instruction > * > > llvm::InterleavedAccessInfo::getInterleaveGroups ( )
inline

Definition at line 637 of file VectorUtils.h.

References llvm::make_range().

◆ hasGroups()

bool llvm::InterleavedAccessInfo::hasGroups ( ) const
inline

Returns true if we have any interleave groups.

Definition at line 651 of file VectorUtils.h.

Referenced by llvm::AArch64TTIImpl::preferPredicateOverEpilogue().

◆ invalidateGroups()

bool llvm::InterleavedAccessInfo::invalidateGroups ( )
inline

Invalidate groups, e.g., in case all blocks in loop will be predicated contrary to original assumption.

Although we currently prevent group formation for predicated accesses, we may be able to relax this limitation in the future once we handle more complicated blocks. Returns true if any groups were invalidated.

Definition at line 607 of file VectorUtils.h.

References assert(), and Ptr.

Referenced by llvm::LoopVectorizationPlanner::plan(), and ~InterleavedAccessInfo().

◆ invalidateGroupsRequiringScalarEpilogue()

void InterleavedAccessInfo::invalidateGroupsRequiringScalarEpilogue ( )

Invalidate groups that require a scalar epilogue (due to gaps).

This can happen when optimizing for size forbids a scalar epilogue, and the gap cannot be filtered by masking the load/store.

Definition at line 1453 of file VectorUtils.cpp.

References assert(), llvm::dbgs(), LLVM_DEBUG, llvm::make_early_inc_range(), and requiresScalarEpilogue().

Referenced by llvm::LoopVectorizationCostModel::computeMaxVF().

◆ isInterleaved()

bool llvm::InterleavedAccessInfo::isInterleaved ( Instruction Instr) const
inline

Check if Instr belongs to any interleave group.

Definition at line 624 of file VectorUtils.h.

Referenced by analyzeInterleaving(), and llvm::LoopVectorizationCostModel::isAccessInterleaved().

◆ requiresScalarEpilogue()

bool llvm::InterleavedAccessInfo::requiresScalarEpilogue ( ) const
inline

Returns true if an interleaved group that may access memory out-of-bounds requires a scalar epilogue iteration for correctness.

Definition at line 643 of file VectorUtils.h.

Referenced by invalidateGroupsRequiringScalarEpilogue(), and llvm::LoopVectorizationCostModel::requiresScalarEpilogue().


The documentation for this class was generated from the following files: