LLVM 23.0.0git
llvm::InnerLoopVectorizer Class Reference

InnerLoopVectorizer vectorizes loops which contain only one basic block to a specified vectorization factor (VF). More...

Inheritance diagram for llvm::InnerLoopVectorizer:
[legend]

Public Member Functions

 InnerLoopVectorizer (Loop *OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo *LI, DominatorTree *DT, const TargetTransformInfo *TTI, AssumptionCache *AC, ElementCount VecWidth, unsigned UnrollFactor, LoopVectorizationCostModel *CM, GeneratedRTChecks &RTChecks, VPlan &Plan)
virtual ~InnerLoopVectorizer ()=default
virtual BasicBlockcreateVectorizedLoopSkeleton ()
 Creates a basic block for the scalar preheader.
void fixVectorizedLoop (VPTransformState &State)
 Fix the vectorized code, taking care of header phi's, and more.
void fixNonInductionPHIs (VPTransformState &State)
 Fix the non-induction PHIs in Plan.

Protected Member Functions

BasicBlockcreateScalarPreheader (StringRef Prefix)
 Create and return a new IR basic block for the scalar preheader whose name is prefixed with Prefix.
virtual void printDebugTracesAtStart ()
 Allow subclasses to override and print debug traces before/after vplan execution, when trace information is requested.
virtual void printDebugTracesAtEnd ()

Protected Attributes

LoopOrigLoop
 The original loop.
PredicatedScalarEvolutionPSE
 A wrapper around ScalarEvolution used to add runtime SCEV checks.
LoopInfoLI
 Loop Info.
DominatorTreeDT
 Dominator Tree.
const TargetTransformInfoTTI
 Target Transform Info.
AssumptionCacheAC
 Assumption Cache.
ElementCount VF
 The vectorization SIMD factor to use.
unsigned UF
 The vectorization unroll factor to use.
IRBuilder Builder
 The builder that we use.
LoopVectorizationCostModelCost
 The profitablity analysis.
GeneratedRTChecks & RTChecks
 Structure to hold information about generated runtime checks, responsible for cleaning the checks, if vectorization turns out unprofitable.
VPlanPlan
VPBasicBlockVectorPHVPBB
 The vector preheader block of Plan, used as target for check blocks introduced during skeleton creation.

Friends

class LoopVectorizationPlanner

Detailed Description

InnerLoopVectorizer vectorizes loops which contain only one basic block to a specified vectorization factor (VF).

This class performs the widening of scalars into vectors, or multiple scalars. This class also implements the following features:

  • It inserts an epilogue loop for handling loops that don't have iteration counts that are known to be a multiple of the vectorization factor.
  • It handles the code generation for reduction variables.
  • Scalarization (implementation using scalars) of un-vectorizable instructions. InnerLoopVectorizer does not perform any vectorization-legality checks, and relies on the caller to check for the different legality aspects. The InnerLoopVectorizer relies on the LoopVectorizationLegality class to provide information about the induction and reduction variables that were found to a given vectorization factor.

Definition at line 546 of file LoopVectorize.cpp.

Constructor & Destructor Documentation

◆ InnerLoopVectorizer()

llvm::InnerLoopVectorizer::InnerLoopVectorizer ( Loop * OrigLoop,
PredicatedScalarEvolution & PSE,
LoopInfo * LI,
DominatorTree * DT,
const TargetTransformInfo * TTI,
AssumptionCache * AC,
ElementCount VecWidth,
unsigned UnrollFactor,
LoopVectorizationCostModel * CM,
GeneratedRTChecks & RTChecks,
VPlan & Plan )
inline

◆ ~InnerLoopVectorizer()

virtual llvm::InnerLoopVectorizer::~InnerLoopVectorizer ( )
virtualdefault

Member Function Documentation

◆ createScalarPreheader()

BasicBlock * InnerLoopVectorizer::createScalarPreheader ( StringRef Prefix)
protected

Create and return a new IR basic block for the scalar preheader whose name is prefixed with Prefix.

Definition at line 2331 of file LoopVectorize.cpp.

References assert(), Cost, DT, llvm::BasicBlock::getTerminator(), LI, OrigLoop, llvm::SplitBlock(), and VF.

Referenced by llvm::EpilogueVectorizerEpilogueLoop::createVectorizedLoopSkeleton(), and createVectorizedLoopSkeleton().

◆ createVectorizedLoopSkeleton()

BasicBlock * InnerLoopVectorizer::createVectorizedLoopSkeleton ( )
virtual

Creates a basic block for the scalar preheader.

Both EpilogueVectorizerMainLoop and EpilogueVectorizerEpilogueLoop overwrite the method to create additional blocks and checks needed for epilogue vectorization.

Reimplemented in llvm::EpilogueVectorizerEpilogueLoop.

Definition at line 2369 of file LoopVectorize.cpp.

References createScalarPreheader(), and llvm::BasicBlock::getSinglePredecessor().

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ fixNonInductionPHIs()

void InnerLoopVectorizer::fixNonInductionPHIs ( VPTransformState & State)

◆ fixVectorizedLoop()

void InnerLoopVectorizer::fixVectorizedLoop ( VPTransformState & State)

Fix the vectorized code, taking care of header phi's, and more.

Definition at line 2499 of file LoopVectorize.cpp.

References fixNonInductionPHIs(), llvm::vputils::getFirstLoopHeader(), and legacyCSE().

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ printDebugTracesAtEnd()

virtual void llvm::InnerLoopVectorizer::printDebugTracesAtEnd ( )
inlineprotectedvirtual

◆ printDebugTracesAtStart()

virtual void llvm::InnerLoopVectorizer::printDebugTracesAtStart ( )
inlineprotectedvirtual

Allow subclasses to override and print debug traces before/after vplan execution, when trace information is requested.

Reimplemented in llvm::EpilogueVectorizerEpilogueLoop, and llvm::EpilogueVectorizerMainLoop.

Definition at line 583 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ LoopVectorizationPlanner

friend class LoopVectorizationPlanner
friend

Definition at line 575 of file LoopVectorize.cpp.

References LoopVectorizationPlanner.

Referenced by LoopVectorizationPlanner.

Member Data Documentation

◆ AC

◆ Builder

IRBuilder llvm::InnerLoopVectorizer::Builder
protected

The builder that we use.

Definition at line 615 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationPlanner::executePlan(), fixNonInductionPHIs(), and InnerLoopVectorizer().

◆ Cost

LoopVectorizationCostModel* llvm::InnerLoopVectorizer::Cost
protected

The profitablity analysis.

Definition at line 620 of file LoopVectorize.cpp.

Referenced by createScalarPreheader(), and InnerLoopVectorizer().

◆ DT

◆ LI

◆ OrigLoop

◆ Plan

◆ PSE

PredicatedScalarEvolution& llvm::InnerLoopVectorizer::PSE
protected

A wrapper around ScalarEvolution used to add runtime SCEV checks.

Applies dynamic knowledge to simplify SCEV expressions and converts them to a more usable form.

Definition at line 592 of file LoopVectorize.cpp.

Referenced by llvm::EpilogueVectorizerEpilogueLoop::EpilogueVectorizerEpilogueLoop(), llvm::EpilogueVectorizerMainLoop::EpilogueVectorizerMainLoop(), llvm::InnerLoopAndEpilogueVectorizer::InnerLoopAndEpilogueVectorizer(), and InnerLoopVectorizer().

◆ RTChecks

GeneratedRTChecks& llvm::InnerLoopVectorizer::RTChecks
protected

Structure to hold information about generated runtime checks, responsible for cleaning the checks, if vectorization turns out unprofitable.

Definition at line 624 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationPlanner::executePlan(), and InnerLoopVectorizer().

◆ TTI

const TargetTransformInfo* llvm::InnerLoopVectorizer::TTI
protected

◆ UF

unsigned llvm::InnerLoopVectorizer::UF
protected

The vectorization unroll factor to use.

Each scalar is vectorized to this many different vector instructions.

Definition at line 612 of file LoopVectorize.cpp.

Referenced by InnerLoopVectorizer().

◆ VectorPHVPBB

VPBasicBlock* llvm::InnerLoopVectorizer::VectorPHVPBB
protected

The vector preheader block of Plan, used as target for check blocks introduced during skeleton creation.

Definition at line 630 of file LoopVectorize.cpp.

Referenced by InnerLoopVectorizer().

◆ VF

ElementCount llvm::InnerLoopVectorizer::VF
protected

The vectorization SIMD factor to use.

Each vector will have this many vector elements.

Definition at line 608 of file LoopVectorize.cpp.

Referenced by createScalarPreheader(), and InnerLoopVectorizer().


The documentation for this class was generated from the following file: