ARMLowOverheadLoops.cpp File Reference

Finalize v8.1-m low-overhead loops by converting the associated pseudo instructions into machine operations. More...

#include "ARM.h"
#include "ARMBaseInstrInfo.h"
#include "ARMBasicBlockInfo.h"
#include "ARMSubtarget.h"
#include "MVETailPredUtils.h"
#include "Thumb2InstrInfo.h"
#include "llvm/ADT/SetVector.h"
#include "llvm/CodeGen/LivePhysRegs.h"
#include "llvm/CodeGen/MachineFrameInfo.h"
#include "llvm/CodeGen/MachineFunctionPass.h"
#include "llvm/CodeGen/MachineLoopInfo.h"
#include "llvm/CodeGen/MachineLoopUtils.h"
#include "llvm/CodeGen/MachineRegisterInfo.h"
#include "llvm/CodeGen/Passes.h"
#include "llvm/CodeGen/ReachingDefAnalysis.h"
#include "llvm/MC/MCInstrDesc.h"

Go to the source code of this file.

Macros
#define	DEBUG_TYPE   "arm-low-overhead-loops"
#define	ARM_LOW_OVERHEAD_LOOPS_NAME   "ARM Low Overhead Loops pass"

Functions
static bool	isVectorPredicated (MachineInstr *MI)
static bool	isVectorPredicate (MachineInstr *MI)
static bool	hasVPRUse (MachineInstr &MI)
static bool	isDomainMVE (MachineInstr *MI)
static int	getVecSize (const MachineInstr &MI)
static bool	shouldInspect (MachineInstr &MI)
static bool	isHorizontalReduction (const MachineInstr &MI)
	INITIALIZE_PASS (ARMLowOverheadLoops, DEBUG_TYPE, ARM_LOW_OVERHEAD_LOOPS_NAME, false, false) static bool TryRemove(MachineInstr *MI
	if (!RDI.isSafeToRemove(MI, Uses, Ignore)) return false
	if (WontCorruptITs(Uses, RDI))
static bool	isRegInClass (const MachineOperand &MO, const TargetRegisterClass *Class)
static bool	retainsPreviousHalfElement (const MachineInstr &MI)
static bool	producesDoubleWidthResult (const MachineInstr &MI)
static bool	canGenerateNonZeros (const MachineInstr &MI)
static bool	producesFalseLanesZero (MachineInstr &MI, const TargetRegisterClass *QPRs, const ReachingDefInfo &RDI, InstSet &FalseLanesZero)
static bool	ValidateMVEStore (MachineInstr *MI, MachineLoop *ML)

Variables
static cl::opt< bool >	DisableTailPredication ("arm-loloops-disable-tailpred", cl::Hidden, cl::desc("Disable tail-predication in the ARM LowOverheadLoop pass"), cl::init(false))
static cl::opt< bool >	DisableOmitDLS ("arm-disable-omit-dls", cl::Hidden, cl::desc("Disable omitting 'dls lr, lr' instructions"), cl::init(false))
ReachingDefInfo &	RDI
ReachingDefInfo InstSet &	ToRemove
ReachingDefInfo InstSet InstSet &	Ignore
SmallPtrSet< MachineInstr *, 2 >	Uses
return	false

Detailed Description

Finalize v8.1-m low-overhead loops by converting the associated pseudo instructions into machine operations.

The expectation is that the loop contains three pseudo instructions:

• t2*LoopStart - placed in the preheader or pre-preheader. The do-loop form should be in the preheader, whereas the while form should be in the preheaders only predecessor.
• t2LoopDec - placed within in the loop body.
• t2LoopEnd - the loop latch terminator.

In addition to this, we also look for the presence of the VCTP instruction, which determines whether we can generated the tail-predicated low-overhead loop form.

Assumptions and Dependencies: Low-overhead loops are constructed and executed using a setup instruction: DLS, WLS, DLSTP or WLSTP and an instruction that loops back: LE or LETP. WLS(TP) and LE(TP) are branching instructions with a (large) limited range but fixed polarity: WLS can only branch forwards and LE can only branch backwards. These restrictions mean that this pass is dependent upon block layout and block sizes, which is why it's the last pass to run. The same is true for ConstantIslands, but this pass does not increase the size of the basic blocks, nor does it change the CFG. Instructions are mainly removed during the transform and pseudo instructions are replaced by real ones. In some cases, when we have to revert to a 'normal' loop, we have to introduce multiple instructions for a single pseudo (see RevertWhile and RevertLoopEnd). To handle this situation, t2WhileLoopStartLR and t2LoopEnd are defined to be as large as this maximum sequence of replacement instructions.

A note on VPR.P0 (the lane mask): VPT, VCMP, VPNOT and VCTP won't overwrite VPR.P0 when they update it in a "VPT Active" context (which includes low-overhead loops and vpt blocks). They will simply "and" the result of their calculation with the current value of VPR.P0. You can think of it like this:

/// if VPT active:    ; Between a DLSTP/LETP, or for predicated instrs
///   VPR.P0 &= Value
/// else
///   VPR.P0 = Value
/// 

When we're inside the low-overhead loop (between DLSTP and LETP), we always fall in the "VPT active" case, so we can consider that all VPR writes by one of those instruction is actually a "and".

Definition in file ARMLowOverheadLoops.cpp.

Macro Definition Documentation

ARM_LOW_OVERHEAD_LOOPS_NAME

#define ARM_LOW_OVERHEAD_LOOPS_NAME   "ARM Low Overhead Loops pass"

Definition at line 74 of file ARMLowOverheadLoops.cpp.

Referenced by INITIALIZE_PASS().

DEBUG_TYPE

#define DEBUG_TYPE   "arm-low-overhead-loops"

Definition at line 73 of file ARMLowOverheadLoops.cpp.

Function Documentation

canGenerateNonZeros()

bool canGenerateNonZeros ( const MachineInstr & MI )

static

Definition at line 829 of file ARMLowOverheadLoops.cpp.

References MI, and producesDoubleWidthResult().

Referenced by producesFalseLanesZero().

getVecSize()

int getVecSize ( const MachineInstr & MI )

static

Definition at line 104 of file ARMLowOverheadLoops.cpp.

References MI, llvm::ARMII::VecSize, and llvm::ARMII::VecSizeShift.

hasVPRUse()

bool hasVPRUse ( MachineInstr & MI )

static

Definition at line 95 of file ARMLowOverheadLoops.cpp.

References MI.

Referenced by shouldInspect().

if() [1/2]

if

(

!RDI.

isSafeToRemoveMI, Uses, Ignore

)

References Ignore, MI, RDI, and Uses.

if() [2/2]

if

(

WontCorruptITs(Uses, RDI)

)

Definition at line 576 of file ARMLowOverheadLoops.cpp.

References llvm::dbgs(), if(), LLVM_DEBUG, MI, RDI, ToRemove, and Uses.

INITIALIZE_PASS()

INITIALIZE_PASS	(	ARMLowOverheadLoops	,
		DEBUG_TYPE	,
		ARM_LOW_OVERHEAD_LOOPS_NAME	,
		false	,
		false	)

References ARM_LOW_OVERHEAD_LOOPS_NAME, and DEBUG_TYPE.

isDomainMVE()

bool isDomainMVE ( MachineInstr * MI )

static

Definition at line 99 of file ARMLowOverheadLoops.cpp.

References llvm::ARMII::DomainMask, llvm::ARMII::DomainMVE, and MI.

Referenced by shouldInspect().

isHorizontalReduction()

bool isHorizontalReduction ( const MachineInstr & MI )

static

Definition at line 116 of file ARMLowOverheadLoops.cpp.

References llvm::ARMII::HorizontalReduction, and MI.

Referenced by producesFalseLanesZero().

isRegInClass()

bool isRegInClass ( const MachineOperand & MO, const TargetRegisterClass * Class )

static

Definition at line 802 of file ARMLowOverheadLoops.cpp.

References llvm::MachineOperand::getReg(), and llvm::MachineOperand::isReg().

Referenced by producesFalseLanesZero().

isVectorPredicate()

bool isVectorPredicate ( MachineInstr * MI )

static

Definition at line 91 of file ARMLowOverheadLoops.cpp.

References MI.

Referenced by shouldInspect().

isVectorPredicated()

bool isVectorPredicated ( MachineInstr * MI )

static

Definition at line 86 of file ARMLowOverheadLoops.cpp.

References llvm::findFirstVPTPredOperandIdx(), and MI.

Referenced by producesFalseLanesZero().

producesDoubleWidthResult()

bool producesDoubleWidthResult ( const MachineInstr & MI )

static

Definition at line 819 of file ARMLowOverheadLoops.cpp.

References llvm::ARMII::DoubleWidthResult, and MI.

Referenced by canGenerateNonZeros().

producesFalseLanesZero()

bool producesFalseLanesZero ( MachineInstr & MI, const TargetRegisterClass * QPRs, const ReachingDefInfo & RDI, InstSet & FalseLanesZero )

static

Definition at line 858 of file ARMLowOverheadLoops.cpp.

References canGenerateNonZeros(), llvm::dbgs(), llvm::SmallPtrSetImplBase::empty(), llvm::findFirstVPTPredOperandIdx(), llvm::MachineOperand::getOperandNo(), llvm::MachineOperand::getReg(), isHorizontalReduction(), llvm::MachineOperand::isReg(), isRegInClass(), llvm::MachineOperand::isUse(), isVectorPredicated(), LLVM_DEBUG, MI, and RDI.

retainsPreviousHalfElement()

bool retainsPreviousHalfElement ( const MachineInstr & MI )

static

Definition at line 810 of file ARMLowOverheadLoops.cpp.

References MI, and llvm::ARMII::RetainsPreviousHalfElement.

shouldInspect()

bool shouldInspect ( MachineInstr & MI )

static

Definition at line 110 of file ARMLowOverheadLoops.cpp.

References hasVPRUse(), isDomainMVE(), isVectorPredicate(), and MI.

ValidateMVEStore()

bool ValidateMVEStore ( MachineInstr * MI, MachineLoop * ML )

static

Definition at line 1120 of file ARMLowOverheadLoops.cpp.

References llvm::SmallPtrSetImpl< PtrType >::contains(), llvm::dyn_cast(), llvm::PseudoSourceValue::FixedStack, I, llvm::SmallPtrSetImpl< PtrType >::insert(), llvm::is_contained(), llvm::PseudoSourceValue::kind(), MI, ML, llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::SmallVectorTemplateCommon< T, typename >::size(), and llvm::MachineBasicBlock::successors().

Variable Documentation

DisableOmitDLS

cl::opt< bool > DisableOmitDLS("arm-disable-omit-dls", cl::Hidden, cl::desc("Disable omitting 'dls lr, lr' instructions"), cl::init(false)) ( "arm-disable-omit-dls" , cl::Hidden , cl::desc("Disable omitting 'dls lr, lr' instructions") , cl::init(false)  )

static

DisableTailPredication

cl::opt< bool > DisableTailPredication("arm-loloops-disable-tailpred", cl::Hidden, cl::desc("Disable tail-predication in the ARM LowOverheadLoop pass"), cl::init(false)) ( "arm-loloops-disable-tailpred" , cl::Hidden , cl::desc("Disable tail-predication in the ARM LowOverheadLoop pass") , cl::init(false)  )

static

false

return false

Definition at line 592 of file ARMLowOverheadLoops.cpp.

Ignore

ReachingDefInfo InstSet InstSet& Ignore

Definition at line 528 of file ARMLowOverheadLoops.cpp.

Referenced by if(), llvm::ReachingDefInfo::isSafeToDefRegAt(), llvm::ReachingDefInfo::isSafeToDefRegAt(), llvm::ReachingDefInfo::isSafeToRemove(), llvm::ReachingDefInfo::isSafeToRemove(), llvm::sys::unicode::startsWith(), and llvm::ReachingDefInfo::~ReachingDefInfo().

RDI

ReachingDefInfo& RDI

Definition at line 527 of file ARMLowOverheadLoops.cpp.

Referenced by llvm::getX86SubSuperRegister(), if(), if(), producesFalseLanesZero(), llvm::ReachingDefAnalysis::run(), and llvm::ReachingDefPrinterPass::run().

ToRemove

ReachingDefInfo InstSet& ToRemove

Examples

/home/buildbot/as-worker-4/publish-doxygen-docs/llvm-project/llvm/include/llvm/Transforms/Utils/Local.h.

Definition at line 527 of file ARMLowOverheadLoops.cpp.

Referenced by llvm::AA::PointerInfo::State::addAccess(), analyzeAndPruneOutputBlocks(), llvm::calculateRegisterUsageForPlan(), checkAndReplaceCmp(), checkAndReplaceCondition(), checkAndReplaceMinMax(), checkOrAndOpImpliedByOther(), llvm::VPlanTransforms::convertToConcreteRecipes(), downcastI64toI32InsertExtractElements(), llvm::EliminateDuplicatePHINodes(), llvm::EliminateDuplicatePHINodes(), EliminateDuplicatePHINodesNaiveImpl(), EliminateDuplicatePHINodesSetBasedImpl(), findBasePointer(), fixI8UseChain(), llvm::objcopy::coff::handleArgs(), if(), llvm::ReachingDefInfo::isSafeToRemove(), llvm::ReachingDefInfo::isSafeToRemove(), legalizeFreeze(), legalizeGetHighLowi64Bytes(), legalizeMemCpy(), legalizeMemSet(), legalizeScalarLoadStoreOnArrays(), lowerAllowChecks(), llvm::SPIRV::RequirementHandler::removeCapabilityIf(), llvm::objcopy::macho::Object::removeLoadCommands(), removeNotes(), llvm::objcopy::dxbc::Object::removeParts(), llvm::objcopy::elf::DynamicRelocationSection::removeSectionReferences(), llvm::objcopy::elf::GroupSection::removeSectionReferences(), llvm::objcopy::elf::RelocationSection::removeSectionReferences(), llvm::objcopy::elf::Section::removeSectionReferences(), llvm::objcopy::elf::SymbolTableSection::removeSectionReferences(), llvm::objcopy::coff::Object::removeSections(), llvm::objcopy::elf::Object::removeSections(), llvm::objcopy::macho::Object::removeSections(), llvm::objcopy::wasm::Object::removeSections(), llvm::objcopy::coff::Object::removeSymbols(), llvm::objcopy::elf::GroupSection::removeSymbols(), llvm::objcopy::elf::Object::removeSymbols(), llvm::objcopy::elf::RelocationSection::removeSymbols(), llvm::objcopy::elf::SymbolTableSection::removeSymbols(), llvm::objcopy::macho::SymbolTable::removeSymbols(), removeUnreachableFunctions(), replaceSubOverflowUses(), llvm::VPlanTransforms::replicateByVF(), sinkMinMaxInBB(), llvm::JumpThreadingPass::threadGuard(), llvm::DbgValueHistoryMap::trimLocationRanges(), llvm::orc::StaticLibraryDefinitionGenerator::tryToGenerate(), tryToSimplifyOverflowMath(), upcastI8AllocasAndUses(), updateFnegToFsub(), llvm::ReachingDefInfo::~ReachingDefInfo(), and llvm::objcopy::elf::SectionBase::~SectionBase().

Uses

SmallPtrSet<MachineInstr *, 2> Uses

Definition at line 572 of file ARMLowOverheadLoops.cpp.