|
LLVM 24.0.0git
|
#include "Target/AMDGPU/AMDGPUTargetTransformInfo.h"
Public Types | |
| enum class | KnownIEEEMode { Unknown , On , Off } |
Definition at line 65 of file AMDGPUTargetTransformInfo.h.
|
strong |
| Enumerator | |
|---|---|
| Unknown | |
| On | |
| Off | |
Definition at line 325 of file AMDGPUTargetTransformInfo.h.
|
explicit |
Definition at line 238 of file AMDGPUTargetTransformInfo.cpp.
References const, F, llvm::TargetTransformInfoImplBase::getDataLayout(), llvm::DenormalMode::getPreserveSign(), and Mode.
|
inlineoverridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 209 of file AMDGPUTargetTransformInfo.h.
References llvm::AMDGPU::addrspacesMayAlias().
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2015 of file AMDGPUTargetTransformInfo.cpp.
References adjustInliningThresholdUsingCallee(), llvm::BasicTTIImplBase< GCNTTIImpl >::DL, getCallArgsTotalAllocaSize(), and llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getFixedValue().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1892 of file AMDGPUTargetTransformInfo.cpp.
References llvm::BasicTTIImplBase< GCNTTIImpl >::areInlineCompatible(), llvm::dbgs(), llvm::TargetMachine::getSubtargetImpl(), llvm::SIModeRegisterDefaults::isInlineCompatible(), and LLVM_DEBUG.
|
inlineoverridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 225 of file AMDGPUTargetTransformInfo.h.
References llvm::AMDGPUAS::LOCAL_ADDRESS, llvm::AMDGPUAS::PRIVATE_ADDRESS, and llvm::AMDGPUAS::REGION_ADDRESS.
| bool GCNTTIImpl::canSimplifyLegacyMulToMul | ( | const Instruction & | I, |
| const Value * | Op0, | ||
| const Value * | Op1, | ||
| InstCombiner & | IC ) const |
Definition at line 446 of file AMDGPUInstCombineIntrinsic.cpp.
References llvm::computeKnownFPClass(), llvm::fcInf, llvm::fcNan, llvm::fcSubnormal, llvm::fcZero, llvm::InstCombiner::getSimplifyQuery(), llvm::SimplifyQuery::getWithInstruction(), I, llvm::APFloatBase::IEEEsingle(), llvm::KnownFPClass::isKnownNeverInfOrNaN(), and llvm::KnownFPClass::isKnownNeverLogicalZero().
Referenced by instCombineIntrinsic().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1551 of file AMDGPUTargetTransformInfo.cpp.
References llvm::SmallVectorTemplateBase< T, bool >::push_back().
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 2121 of file AMDGPUTargetTransformInfo.cpp.
References F, and llvm::AMDGPU::getMaxNumWorkGroups().
| GCNTTIImpl::KnownIEEEMode GCNTTIImpl::fpenvIEEEMode | ( | const Instruction & | I | ) | const |
Return KnownIEEEMode::On if we know if the use context can assume "amdgpu-ieee"="true" and KnownIEEEMode::Off if we can assume "amdgpu-ieee"="false".
Definition at line 2138 of file AMDGPUTargetTransformInfo.cpp.
References F, llvm::Attribute::getValueAsBool(), I, llvm::AMDGPU::isShader(), llvm::Attribute::isValid(), Off, On, and Unknown.
Referenced by getIntrinsicInstrCost(), and instCombineIntrinsic().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 527 of file AMDGPUTargetTransformInfo.cpp.
References llvm::ISD::ADD, llvm::ISD::AND, llvm::dyn_cast(), llvm::ISD::FADD, llvm::ISD::FDIV, llvm::ISD::FMUL, llvm::ISD::FNEG, llvm::ISD::FREM, llvm::ISD::FSUB, llvm::BasicTTIImplBase< GCNTTIImpl >::getArithmeticInstrCost(), getMaximumVF(), llvm::Value::getType(), llvm::Instruction::hasApproxFunc(), llvm::Value::hasOneUse(), isFusedFMul(), llvm::PatternMatch::m_FPOne(), llvm::PatternMatch::match(), llvm::ISD::MUL, llvm::ISD::OR, llvm::ISD::SHL, llvm::ISD::SRA, llvm::ISD::SRL, llvm::ISD::SUB, llvm::TargetTransformInfo::TCC_Free, llvm::Instruction::user_begin(), and llvm::ISD::XOR.
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1247 of file AMDGPUTargetTransformInfo.cpp.
References llvm::BasicTTIImplBase< GCNTTIImpl >::DL, llvm::BasicTTIImplBase< GCNTTIImpl >::getArithmeticReductionCost(), getNumberOfPackedMaskElts(), llvm::EVT::getScalarSizeInBits(), llvm::InstructionCost, MaskPackCostPerElt, and llvm::TargetTransformInfo::requiresOrderedReduction().
|
overridevirtual |
Data cache line size for LoopDataPrefetch pass. Has no use before GFX12.
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2107 of file AMDGPUTargetTransformInfo.cpp.
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2026 of file AMDGPUTargetTransformInfo.cpp.
References llvm::BasicTTIImplBase< GCNTTIImpl >::DL, llvm::AllocaInst::getAllocationSize(), getCallArgsTotalAllocaSize(), llvm::CallBase::getCalledFunction(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getFixedValue(), getInliningThresholdMultiplier(), and llvm::none_of().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1003 of file AMDGPUTargetTransformInfo.cpp.
References AbstractManglingParser< Derived, Alloc >::Ops, llvm::AMDGPUAS::BUFFER_FAT_POINTER, llvm::AMDGPUAS::BUFFER_STRIDED_POINTER, llvm::AMDGPUAS::CONSTANT_ADDRESS, llvm::AMDGPUAS::CONSTANT_ADDRESS_32BIT, llvm::divideCeil(), llvm::BasicTTIImplBase< GCNTTIImpl >::DL, llvm::dyn_cast(), llvm::ISD::FP_TO_SINT, llvm::ISD::FP_TO_UINT, llvm::FixedVectorType::get(), llvm::BasicTTIImplBase< GCNTTIImpl >::getCastInstrCost(), getCastInstrCost(), llvm::Type::getFloatTy(), getNumberOfPackedMaskElts(), llvm::AMDGPUAS::GLOBAL_ADDRESS, I, llvm::InstructionCost, llvm::isa(), llvm::Type::isBFloatTy(), llvm::Type::isDoubleTy(), llvm::Type::isFloatTy(), llvm::Type::isHalfTy(), llvm::isPowerOf2_32(), llvm::EVT::isVector(), llvm::Load, MaskPackCostPerElt, MaskUnpackCostPerElt, llvm::ISD::SINT_TO_FP, and llvm::ISD::UINT_TO_FP.
Referenced by getCastInstrCost().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 950 of file AMDGPUTargetTransformInfo.cpp.
References assert(), llvm::dyn_cast_or_null(), llvm::BasicTTIImplBase< GCNTTIImpl >::getCFInstrCost(), I, llvm::TargetTransformInfo::TCK_CodeSize, and llvm::TargetTransformInfo::TCK_SizeAndLatency.
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 978 of file AMDGPUTargetTransformInfo.cpp.
References llvm::BasicTTIImplBase< GCNTTIImpl >::getCmpSelInstrCost(), I, and llvm::TargetTransformInfo::TCK_SizeAndLatency.
|
inlineoverridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 213 of file AMDGPUTargetTransformInfo.h.
References llvm::AMDGPUAS::FLAT_ADDRESS.
|
inlineoverridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 282 of file AMDGPUTargetTransformInfo.h.
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 2010 of file AMDGPUTargetTransformInfo.cpp.
References llvm::TargetTransformInfoImplBase::getInliningLastCallToStaticBonus(), and getInliningThresholdMultiplier().
|
inlineoverridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 277 of file AMDGPUTargetTransformInfo.h.
Referenced by getCallerAllocaCost(), and getInliningLastCallToStaticBonus().
|
overridevirtual |
Get intrinsic cost based on arguments.
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 709 of file AMDGPUTargetTransformInfo.cpp.
References AbstractManglingParser< Derived, Alloc >::NumOps, llvm::any_of(), llvm::FastMathFlags::approxFunc(), llvm::equal_to(), fpenvIEEEMode(), llvm::IntrinsicCostAttributes::getFlags(), llvm::IntrinsicCostAttributes::getID(), llvm::IntrinsicCostAttributes::getInst(), llvm::BasicTTIImplBase< GCNTTIImpl >::getIntrinsicInstrCost(), llvm::IntrinsicCostAttributes::getReturnType(), II, intrinsicHasPackedVectorBenefit(), and Off.
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 326 of file AMDGPUTargetTransformInfo.cpp.
References llvm::AMDGPUAS::BUFFER_FAT_POINTER, llvm::AMDGPUAS::BUFFER_RESOURCE, llvm::AMDGPUAS::BUFFER_STRIDED_POINTER, llvm::AMDGPUAS::CONSTANT_ADDRESS, llvm::AMDGPUAS::CONSTANT_ADDRESS_32BIT, llvm::AMDGPUAS::GLOBAL_ADDRESS, and llvm::AMDGPUAS::PRIVATE_ADDRESS.
Referenced by getMemoryOpCost().
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 305 of file AMDGPUTargetTransformInfo.cpp.
References llvm::Type::getScalarSizeInBits().
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 282 of file AMDGPUTargetTransformInfo.cpp.
Referenced by getArithmeticInstrCost().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 441 of file AMDGPUTargetTransformInfo.cpp.
References llvm::ElementCount::isScalar().
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 368 of file AMDGPUTargetTransformInfo.cpp.
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 372 of file AMDGPUTargetTransformInfo.cpp.
References llvm::FixedVectorType::get(), llvm::Type::getInt32Ty(), llvm::Type::getIntNTy(), llvm::isa(), and llvm::Length.
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 399 of file AMDGPUTargetTransformInfo.cpp.
References llvm::FixedVectorType::get(), llvm::Type::getInt16Ty(), llvm::Type::getInt32Ty(), llvm::Type::getInt64Ty(), llvm::Type::getInt8Ty(), llvm::TargetTransformInfoImplBase::getMemcpyLoopResidualLoweringType(), and llvm::SmallVectorTemplateBase< T, bool >::push_back().
|
overridevirtual |
Account for loads of i8 vector types to have reduced cost.
For example the cost of load 4 i8s values is one is the cost of loading a single i32 value.
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2221 of file AMDGPUTargetTransformInfo.cpp.
References llvm::AMDGPUAS::BUFFER_FAT_POINTER, llvm::commonAlignment(), llvm::AMDGPUAS::CONSTANT_ADDRESS, llvm::AMDGPUAS::CONSTANT_ADDRESS_32BIT, llvm::divideCeil(), llvm::BasicTTIImplBase< GCNTTIImpl >::DL, llvm::dyn_cast(), llvm::AMDGPUAS::FLAT_ADDRESS, getKnownBufferOobProperties(), getLoadStoreVecRegBitWidth(), llvm::BasicTTIImplBase< GCNTTIImpl >::getMemoryOpCost(), getMemoryOpCost(), getNumMemoryInsts(), llvm::Type::getScalarType(), hasUniformPointerOperand(), I, llvm::isa(), llvm::AMDGPU::isExtendedGlobalAddrSpace(), llvm::Type::isFloatingPointTy(), llvm::Type::isIntOrPtrTy(), llvm::isPowerOf2_32(), llvm::AMDGPUAS::LOCAL_ADDRESS, llvm::popcount(), llvm::PowerOf2Ceil(), llvm::AMDGPUAS::PRIVATE_ADDRESS, llvm::AMDGPUAS::REGION_ADDRESS, and llvm::TargetTransformInfo::TCK_Latency.
Referenced by getMemoryOpCost().
|
overridevirtual |
Try to calculate op costs for min/max reduction operations.
| CondTy | Conditional type for the Select instruction. |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1273 of file AMDGPUTargetTransformInfo.cpp.
References llvm::BasicTTIImplBase< GCNTTIImpl >::DL, llvm::BasicTTIImplBase< GCNTTIImpl >::getMinMaxReductionCost(), and llvm::EVT::getScalarSizeInBits().
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 278 of file AMDGPUTargetTransformInfo.cpp.
When counting parts on AMD GPUs, account for i8s being grouped together under a single i32 value.
Otherwise fall back to base implementation.
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2324 of file AMDGPUTargetTransformInfo.cpp.
References llvm::divideCeil(), llvm::dyn_cast(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getFixedValue(), and llvm::BasicTTIImplBase< GCNTTIImpl >::getNumberOfParts().
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 251 of file AMDGPUTargetTransformInfo.cpp.
|
inlineoverridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 296 of file AMDGPUTargetTransformInfo.h.
References llvm::InstructionCost::getInvalid().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2077 of file AMDGPUTargetTransformInfo.cpp.
|
inlineoverridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 128 of file AMDGPUTargetTransformInfo.h.
References assert(), llvm::isPowerOf2_32(), and llvm::TargetTransformInfo::PSK_FastHardware.
|
overridevirtual |
How much before a load we should place the prefetch instruction.
This is currently measured in number of IR instructions.
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2113 of file AMDGPUTargetTransformInfo.cpp.
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 263 of file AMDGPUTargetTransformInfo.cpp.
References llvm::TypeSize::getFixed(), llvm::TypeSize::getScalable(), llvm_unreachable, llvm::TargetTransformInfo::RGK_FixedWidthVector, llvm::TargetTransformInfo::RGK_ScalableVector, and llvm::TargetTransformInfo::RGK_Scalar.
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2353 of file AMDGPUTargetTransformInfo.cpp.
References llvm::AMDGPUAS::FLAT_ADDRESS, llvm::TargetTransformInfoImplBase::getDataLayout(), llvm::TypeSize::getFixed(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getFixedValue(), llvm::BasicTTIImplBase< GCNTTIImpl >::getScalingFactorCost(), llvm::DataLayout::getTypeStoreSize(), llvm::AMDGPU::isExtendedGlobalAddrSpace(), llvm::details::FixedOrScalableQuantity< TypeSize, uint64_t >::isKnownLE(), and llvm::AMDGPUAS::PRIVATE_ADDRESS.
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1636 of file AMDGPUTargetTransformInfo.cpp.
References llvm::Aligned, assert(), llvm::cast(), llvm::divideCeil(), llvm::BasicTTIImplBase< GCNTTIImpl >::DL, llvm::dyn_cast(), llvm::InstructionCost::getInvalid(), llvm::BasicTTIImplBase< GCNTTIImpl >::getShuffleCost(), llvm::Hi, I, llvm::BasicTTIImplBase< GCNTTIImpl >::improveShuffleKindFromMask(), llvm::is_contained(), llvm::isa(), llvm::Lo, llvm::PoisonMaskElem, llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::SmallVectorTemplateCommon< T, typename >::size(), llvm::TargetTransformInfo::SK_Broadcast, llvm::TargetTransformInfo::SK_ExtractSubvector, llvm::TargetTransformInfo::SK_InsertSubvector, llvm::TargetTransformInfo::SK_PermuteSingleSrc, llvm::TargetTransformInfo::SK_Reverse, llvm::TargetTransformInfo::SK_Splice, llvm::TargetTransformInfo::TCK_RecipThroughput, and llvm::AMDGPUSubtarget::VOLCANIC_ISLANDS.
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 316 of file AMDGPUTargetTransformInfo.cpp.
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 451 of file AMDGPUTargetTransformInfo.cpp.
References llvm::dyn_cast(), llvm::CallBase::getArgOperand(), llvm::IntrinsicInst::getIntrinsicID(), and llvm::isValidAtomicOrdering().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2071 of file AMDGPUTargetTransformInfo.cpp.
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 2334 of file AMDGPUTargetTransformInfo.cpp.
References llvm::AlwaysUniform, llvm::Custom, llvm::Default, llvm::dyn_cast(), and llvm::NeverUniform.
|
inlineoverridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1516 of file BasicTTIImpl.h.
|
inlineoverridevirtual |
| ScalarUserAndIdx | encodes the information about extracts from a vector with 'Scalar' being the value being extracted,'User' being the user of the extract(nullptr if user is not known before vectorization) and 'Idx' being the extract lane. |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1505 of file BasicTTIImpl.h.
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1287 of file AMDGPUTargetTransformInfo.cpp.
References llvm::cast(), llvm::BasicTTIImplBase< GCNTTIImpl >::DL, llvm::dyn_cast(), llvm::dyn_cast_or_null(), llvm::BasicTTIImplBase< GCNTTIImpl >::getVectorInstrCost(), llvm::isa(), llvm::isPowerOf2_32(), llvm::TargetTransformInfo::TCC_Free, and llvm::TargetTransformInfo::TCK_RecipThroughput.
Referenced by isProfitableToSinkOperands().
|
inline |
Definition at line 260 of file AMDGPUTargetTransformInfo.h.
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 247 of file AMDGPUTargetTransformInfo.cpp.
References F.
| Instruction * GCNTTIImpl::hoistLaneIntrinsicThroughOperand | ( | InstCombiner & | IC, |
| IntrinsicInst & | II ) const |
Definition at line 659 of file AMDGPUInstCombineIntrinsic.cpp.
References AbstractManglingParser< Derived, Alloc >::Ops, assert(), llvm::InstCombiner::Builder, llvm::Instruction::clone(), llvm::DominatorTree::dominates(), llvm::dyn_cast(), llvm::InstCombiner::getDominatorTree(), llvm::User::getOperand(), llvm::User::getOperandUse(), llvm::Intrinsic::getOrInsertDeclaration(), llvm::ilist_detail::node_parent_access< NodeTy, ParentTy >::getParent(), llvm::Value::hasOneUser(), II, llvm::isa(), llvm::AMDGPU::isTriviallyUniform(), llvm::BasicTTIImplBase< GCNTTIImpl >::isTypeLegal(), rewriteCall(), and llvm::User::setOperand().
Referenced by instCombineIntrinsic().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1203 of file AMDGPUInstCombineIntrinsic.cpp.
References llvm::ConstantRange::add(), llvm::FastMathFlags::allowContract(), assert(), llvm::APFloat::bitcastToAPInt(), llvm::InstCombiner::Builder, C(), C0, C1, Call, canContractSqrtToRsq(), canSimplifyLegacyMulToMul(), llvm::cast(), Changed, llvm::computeConstantRange(), llvm::APFloat::convert(), llvm::Instruction::copyMetadata(), llvm::IRBuilderBase::CreateAShr(), llvm::IRBuilderBase::CreateCall(), llvm::IRBuilderBase::CreateExtractVector(), llvm::IRBuilderBase::CreateFAddFMF(), llvm::IRBuilderBase::CreateFMulFMF(), llvm::IRBuilderBase::CreateICmpNE(), llvm::IRBuilderBase::CreateInsertElement(), llvm::IRBuilderBase::CreateIntrinsic(), llvm::IRBuilderBase::CreateIntrinsicWithoutFolding(), llvm::IRBuilderBase::CreateLShr(), llvm::IRBuilderBase::CreateMaximumNum(), llvm::IRBuilderBase::CreateMaxNum(), llvm::IRBuilderBase::CreateMinimumNum(), llvm::IRBuilderBase::CreateMinNum(), llvm::IRBuilderBase::CreateShl(), llvm::IRBuilderBase::CreateZExt(), llvm::IRBuilderBase::CreateZExtOrTrunc(), defaultComponentBroadcast(), llvm::dyn_cast(), llvm::InstCombiner::eraseInstFromFunction(), llvm::AMDGPU::evaluateRcp(), llvm::Exponent, llvm::APInt::extractBitsAsZExtValue(), llvm::FAdd, llvm::fcAllFlags, fmed3AMDGCN(), llvm::FMul, foldAddIntoDotAccumulator(), llvm::AMDGPU::MFMAScaleFormats::FP4_E2M1, llvm::AMDGPU::MFMAScaleFormats::FP6_E2M3, llvm::AMDGPU::MFMAScaleFormats::FP6_E3M2, llvm::AMDGPU::MFMAScaleFormats::FP8_E4M3, llvm::AMDGPU::MFMAScaleFormats::FP8_E5M2, fpenvIEEEMode(), llvm::frexp(), llvm::FixedVectorType::get(), llvm::PoisonValue::get(), llvm::UndefValue::get(), llvm::AMDGPU::getExponentOnlyScaleArgIdx(), llvm::ConstantInt::getFalse(), llvm::FPMathOperator::getFastMathFlags(), llvm::Type::getHalfTy(), llvm::AMDGPU::getImageDimIntrinsicInfo(), llvm::ConstantFP::getInfinity(), llvm::IRBuilderBase::getInt1(), llvm::IRBuilderBase::getInt64(), llvm::IRBuilderBase::getIntNTy(), llvm::ConstantFP::getNaN(), llvm::Constant::getNullValue(), llvm::Intrinsic::getOrInsertDeclaration(), llvm::APFloat::getQNaN(), llvm::ConstantInt::getSigned(), llvm::InstCombiner::getSimplifyQuery(), llvm::Value::getType(), llvm::ConstantInt::getValue(), llvm::ConstantFP::getValueAPF(), llvm::Attribute::getWithDereferenceableBytes(), llvm::APFloat::getZero(), llvm::ConstantFP::getZero(), llvm::ConstantInt::getZExtValue(), hoistLaneIntrinsicThroughOperand(), I, llvm::APFloatBase::IEEEhalf(), llvm::APFloatBase::IEK_Inf, llvm::APFloatBase::IEK_NaN, II, llvm::ConstantRange::intersectWith(), llvm::isa(), llvm::ConstantRange::isFullSet(), llvm::APFloat::isInfinity(), llvm::APFloat::isNaN(), llvm::APFloat::isPosInfinity(), llvm::APFloat::isSignaling(), llvm::AMDGPU::isTriviallyUniform(), llvm::SimplifyQuery::isUndefValue(), llvm_unreachable, llvm::PatternMatch::m_AnyZeroFP(), llvm::PatternMatch::m_APFloat(), llvm::PatternMatch::m_ConstantFP(), llvm::PatternMatch::m_FPExt(), llvm::PatternMatch::m_Value(), llvm::PatternMatch::m_Zero(), llvm::Make_64(), llvm::APFloat::makeQuiet(), llvm::PatternMatch::match(), matchFPExtFromF16(), llvm::NearestTiesToEven, Off, llvm::Offset, On, llvm::InstCombiner::replaceInstUsesWith(), llvm::InstCombiner::replaceOperand(), llvm::APFloatBase::rmNearestTiesToEven, llvm::APFloatBase::rmTowardZero, llvm::scalbn(), llvm::CallBase::setAttributes(), llvm::CallInst::setTailCallKind(), llvm::Signed, simplifyAMDGCNImageIntrinsic(), simplifyAMDGCNMemoryIntrinsicDemanded(), simplifyDemandedLaneMaskArg(), simplifyExponentOnlyScaleArg(), llvm::InstCombiner::stripSignOnlyFPOps(), Swap, std::swap(), llvm::Value::takeName(), trimTrailingZerosInVector(), tryOptimizeShufflePattern(), Unknown, llvm::AMDGPU::wmmaScaleF8F6F4FormatToNumRegs(), X, Y, and Zero.
| bool GCNTTIImpl::isInlineAsmSourceOfDivergence | ( | const CallBase * | CB, |
| ArrayRef< unsigned > | Indices = {} ) const |
Analyze if the results of inline asm are divergent.
If Indices is empty, this is analyzing the collective result of all output registers. Otherwise, this is only querying a specific result index if this returns multiple registers in a struct.
Definition at line 1350 of file AMDGPUTargetTransformInfo.cpp.
References llvm::BasicTTIImplBase< GCNTTIImpl >::DL, llvm::ArrayRef< T >::empty(), llvm::Instruction::getDataLayout(), llvm::InlineAsm::isOutput, SDValue(), llvm::ArrayRef< T >::size(), and TRI.
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 356 of file AMDGPUTargetTransformInfo.cpp.
References isLegalToVectorizeMemChain().
| bool GCNTTIImpl::isLegalToVectorizeMemChain | ( | unsigned | ChainSizeInBytes, |
| Align | Alignment, | ||
| unsigned | AddrSpace ) const |
Definition at line 343 of file AMDGPUTargetTransformInfo.cpp.
References llvm::AMDGPUAS::PRIVATE_ADDRESS.
Referenced by isLegalToVectorizeLoadChain(), and isLegalToVectorizeStoreChain().
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 362 of file AMDGPUTargetTransformInfo.cpp.
References isLegalToVectorizeMemChain().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2376 of file AMDGPUTargetTransformInfo.cpp.
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2390 of file AMDGPUTargetTransformInfo.cpp.
|
overridevirtual |
Whether it is profitable to sink the operands of an Instruction I to the basic block of I.
This helps using several modifiers (like abs and neg) more often.
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 1811 of file AMDGPUTargetTransformInfo.cpp.
References AbstractManglingParser< Derived, Alloc >::Ops, llvm::any_of(), llvm::cast(), llvm::BasicTTIImplBase< GCNTTIImpl >::DL, llvm::dyn_cast(), llvm::FMul, getFusedFMul(), getVectorInstrCost(), llvm::Value::hasOneUse(), I, llvm::PatternMatch::m_FAbs(), llvm::PatternMatch::m_FNegOrFSubNegZero(), llvm::PatternMatch::m_Value(), llvm::PatternMatch::match(), and llvm::TargetTransformInfo::TCK_RecipThroughput.
| bool GCNTTIImpl::isReadRegisterSourceOfDivergence | ( | const IntrinsicInst * | ReadReg | ) | const |
Definition at line 1386 of file AMDGPUTargetTransformInfo.cpp.
References llvm::cast(), llvm::CallBase::getArgOperand(), llvm::Value::getType(), llvm::MVT::getVT(), and RegName.
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 2400 of file AMDGPUTargetTransformInfo.cpp.
References llvm::cast(), I, and llvm_unreachable.
|
inlineoverridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 200 of file AMDGPUTargetTransformInfo.h.
References llvm::AMDGPU::addrspacesMayAlias().
|
overridevirtual |
Reimplemented from llvm::TargetTransformInfoImplBase.
Definition at line 296 of file AMDGPUTargetTransformInfo.cpp.
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 1568 of file AMDGPUTargetTransformInfo.cpp.
References llvm::PointerType::get(), llvm::Type::getContext(), llvm::ConstantInt::getFalse(), llvm::Intrinsic::getOrInsertDeclaration(), llvm::Type::getPointerAddressSpace(), llvm::ConstantInt::getTrue(), llvm::Value::getType(), llvm::AMDGPUAS::GLOBAL_ADDRESS, II, llvm::IRBuilder(), llvm::AMDGPU::isExtendedGlobalAddrSpace(), llvm::AMDGPUAS::LOCAL_ADDRESS, and llvm::AMDGPUAS::PRIVATE_ADDRESS.
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2395 of file AMDGPUTargetTransformInfo.cpp.
AS. Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2117 of file AMDGPUTargetTransformInfo.cpp.
References llvm::AMDGPU::isFlatGlobalAddrSpace().
| Value * GCNTTIImpl::simplifyAMDGCNLaneIntrinsicDemanded | ( | InstCombiner & | IC, |
| IntrinsicInst & | II, | ||
| const APInt & | DemandedElts, | ||
| APInt & | UndefElts ) const |
Definition at line 2546 of file AMDGPUInstCombineIntrinsic.cpp.
References llvm::InstCombiner::Builder, llvm::APInt::countr_zero(), llvm::IRBuilderBase::CreateCall(), llvm::IRBuilderBase::CreateExtractElement(), llvm::IRBuilderBase::CreateInsertElement(), llvm::IRBuilderBase::CreateShuffleVector(), llvm::dyn_cast(), llvm::FixedVectorType::get(), llvm::PoisonValue::get(), llvm::APInt::getActiveBits(), llvm::IRBuilderBase::getModule(), llvm::Intrinsic::getOrInsertDeclaration(), I, II, and llvm::BasicTTIImplBase< GCNTTIImpl >::isTypeLegal().
Referenced by simplifyDemandedVectorEltsIntrinsic().
| Value * GCNTTIImpl::simplifyAMDGCNMemoryIntrinsicDemanded | ( | InstCombiner & | IC, |
| IntrinsicInst & | II, | ||
| APInt | DemandedElts ) const |
Implement SimplifyDemandedVectorElts for amdgcn buffer and image intrinsics.
The result of simplifying amdgcn image and buffer store intrinsics is updating definitions of the intrinsics vector argument, not Uses of the result like image and buffer loads. Note: This only supports non-TFE/LWE image intrinsic calls; those have struct returns.
Definition at line 2351 of file AMDGPUInstCombineIntrinsic.cpp.
References llvm::CallBase::addParamAttr(), llvm::AMDGPU::GcnBufferFormatInfo::BitsPerComp, llvm::InstCombiner::Builder, llvm::cast(), llvm::commonAlignment(), llvm::Instruction::copyMetadata(), llvm::APInt::countr_zero(), llvm::IRBuilderBase::CreateAdd(), llvm::IRBuilderBase::CreateExtractElement(), llvm::IRBuilderBase::CreateInsertElement(), llvm::IRBuilderBase::CreateIntrinsicWithoutFolding(), llvm::IRBuilderBase::CreateShuffleVector(), llvm::AMDGPU::GcnBufferFormatInfo::Format, llvm::FixedVectorType::get(), llvm::PoisonValue::get(), llvm::APInt::getActiveBits(), llvm::Value::getContext(), llvm::InstCombiner::getDataLayout(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getFixedValue(), llvm::AMDGPU::BufferIntrinsicInfo::getFormatArg(), llvm::AMDGPU::getGcnBufferFormatInfo(), llvm::AMDGPU::BufferIntrinsicInfo::getOffsetArg(), llvm::CallBase::getParamAlign(), getType(), llvm::Value::getType(), llvm::DataLayout::getTypeSizeInBits(), llvm::Attribute::getWithAlignment(), llvm::ConstantInt::getZExtValue(), II, llvm::AMDGPU::BufferIntrinsicInfo::isFormatFromRsrc(), llvm::APInt::isMask(), llvm::isMask_32(), llvm::Intrinsic::isSignatureValid(), llvm::AMDGPU::BufferIntrinsicInfo::isTyped(), llvm::AMDGPU::BufferIntrinsicInfo::Kind, llvm::AMDGPU::lookupBufferIntrinsic(), llvm::AMDGPU::lookupRsrcIntrinsic(), llvm::AMDGPU::GcnBufferFormatInfo::NumComponents, llvm::AMDGPU::GcnBufferFormatInfo::NumFormat, llvm::Offset, llvm::APInt::popcount(), llvm::popcount(), llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::CallBase::removeParamAttr(), llvm::AMDGPU::RsrcIntrinsic::RsrcArg, llvm::CallBase::setAttributes(), llvm::IRBuilderBase::SetInsertPoint(), and llvm::Value::takeName().
Referenced by instCombineIntrinsic(), and simplifyDemandedVectorEltsIntrinsic().
| bool GCNTTIImpl::simplifyDemandedLaneMaskArg | ( | InstCombiner & | IC, |
| IntrinsicInst & | II, | ||
| unsigned | LaneArgIdx ) const |
Simplify a lane index operand (e.g.
llvm.amdgcn.readlane src1).
The instruction only reads the low 5 bits for wave32, and 6 bits for wave64.
Definition at line 565 of file AMDGPUInstCombineIntrinsic.cpp.
References llvm::Value::getType(), II, llvm::Known, llvm::maskTrailingOnes(), and llvm::InstCombiner::SimplifyDemandedBits().
Referenced by instCombineIntrinsic().
|
overridevirtual |
Reimplemented from llvm::BasicTTIImplBase< GCNTTIImpl >.
Definition at line 2612 of file AMDGPUInstCombineIntrinsic.cpp.
References II, llvm::AMDGPU::lookupBufferIntrinsic(), simplifyAMDGCNLaneIntrinsicDemanded(), and simplifyAMDGCNMemoryIntrinsicDemanded().