LLVM
15.0.0git
|
Helper struct to manage generating runtime checks for vectorization. More...
Public Member Functions | |
GeneratedRTChecks (ScalarEvolution &SE, DominatorTree *DT, LoopInfo *LI, const DataLayout &DL) | |
void | Create (Loop *L, const LoopAccessInfo &LAI, const SCEVPredicate &UnionPred, ElementCount VF, unsigned IC) |
Generate runtime checks in SCEVCheckBlock and MemCheckBlock, so we can accurately estimate the cost of the runtime checks. More... | |
~GeneratedRTChecks () | |
Remove the created SCEV & memory runtime check blocks & instructions, if unused. More... | |
BasicBlock * | emitSCEVChecks (BasicBlock *Bypass, BasicBlock *LoopVectorPreHeader, BasicBlock *LoopExitBlock) |
Adds the generated SCEVCheckBlock before LoopVectorPreHeader and adjusts the branches to branch to the vector preheader or Bypass , depending on the generated condition. More... | |
BasicBlock * | emitMemRuntimeChecks (BasicBlock *Bypass, BasicBlock *LoopVectorPreHeader) |
Adds the generated MemCheckBlock before LoopVectorPreHeader and adjusts the branches to branch to the vector preheader or Bypass , depending on the generated condition. More... | |
Helper struct to manage generating runtime checks for vectorization.
The runtime checks are created up-front in temporary blocks to allow better estimating the cost and un-linked from the existing IR. After deciding to vectorize, the checks are moved back. If deciding not to vectorize, the temporary blocks are completely removed.
Definition at line 1909 of file LoopVectorize.cpp.
|
inline |
Definition at line 1932 of file LoopVectorize.cpp.
|
inline |
Remove the created SCEV & memory runtime check blocks & instructions, if unused.
Definition at line 2018 of file LoopVectorize.cpp.
References llvm::SCEVExpanderCleaner::cleanup(), llvm::BasicBlock::eraseFromParent(), llvm::ScalarEvolution::forgetValue(), llvm::SCEVExpander::getSE(), I, llvm::SCEVExpander::isInsertedInstruction(), llvm::make_early_inc_range(), llvm::SCEVExpanderCleaner::markResultUsed(), and llvm::reverse().
|
inline |
Generate runtime checks in SCEVCheckBlock and MemCheckBlock, so we can accurately estimate the cost of the runtime checks.
The blocks are un-linked from the IR and is added back during vector code generation. If there is no vector code generation, the check blocks are removed completely.
Definition at line 1942 of file LoopVectorize.cpp.
References llvm::addDiffRuntimeChecks(), llvm::addRuntimeChecks(), assert(), B, llvm::tgtok::Bits, llvm::DominatorTreeBase< NodeT, IsPostDom >::changeImmediateDominator(), llvm::InnerLoopVectorizer::DT, llvm::Instruction::eraseFromParent(), llvm::DominatorTreeBase< NodeT, IsPostDom >::eraseNode(), llvm::SCEVExpander::expandCodeForPredicate(), llvm::BasicBlock::getContext(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::LoopBase< BlockT, LoopT >::getLoopPreheader(), llvm::LoopAccessInfo::getRuntimePointerChecking(), llvm::BasicBlock::getTerminator(), llvm::SCEVPredicate::isAlwaysTrue(), llvm::InnerLoopVectorizer::LI, llvm::Instruction::moveBefore(), llvm::LoopInfoBase< BlockT, LoopT >::removeBlock(), llvm::Value::replaceAllUsesWith(), llvm::SplitBlock(), and llvm::InnerLoopVectorizer::VF.
Referenced by llvm::LoopVectorizePass::processLoop().
|
inline |
Adds the generated MemCheckBlock before LoopVectorPreHeader
and adjusts the branches to branch to the vector preheader or Bypass
, depending on the generated condition.
Definition at line 2086 of file LoopVectorize.cpp.
References llvm::DominatorTreeBase< NodeT, IsPostDom >::addNewBlock(), llvm::DominatorTreeBase< NodeT, IsPostDom >::changeImmediateDominator(), llvm::BranchInst::Create(), llvm::InnerLoopVectorizer::DT, llvm::LoopInfoBase< BlockT, LoopT >::getLoopFor(), llvm::BasicBlock::getSinglePredecessor(), llvm::BasicBlock::getTerminator(), llvm::InnerLoopVectorizer::LI, llvm::InnerLoopVectorizer::LoopVectorPreHeader, llvm::BasicBlock::moveBefore(), llvm::AArch64CC::PL, llvm::ReplaceInstWithInst(), llvm::Instruction::replaceSuccessorWith(), and llvm::Instruction::setDebugLoc().
Referenced by llvm::InnerLoopVectorizer::emitMemRuntimeChecks().
|
inline |
Adds the generated SCEVCheckBlock before LoopVectorPreHeader
and adjusts the branches to branch to the vector preheader or Bypass
, depending on the generated condition.
Definition at line 2050 of file LoopVectorize.cpp.
References llvm::DominatorTreeBase< NodeT, IsPostDom >::addNewBlock(), llvm::DominatorTreeBase< NodeT, IsPostDom >::changeImmediateDominator(), Cond, llvm::BranchInst::Create(), llvm::InnerLoopVectorizer::DT, llvm::Instruction::eraseFromParent(), llvm::LoopInfoBase< BlockT, LoopT >::getLoopFor(), llvm::BasicBlock::getSinglePredecessor(), llvm::BasicBlock::getTerminator(), llvm::InnerLoopVectorizer::LI, llvm::InnerLoopVectorizer::LoopVectorPreHeader, llvm::BasicBlock::moveBefore(), llvm::AArch64CC::PL, and llvm::ReplaceInstWithInst().
Referenced by llvm::InnerLoopVectorizer::emitSCEVChecks().