clang
3.9.0
|
Token - This structure provides full information about a lexed token. More...
#include <Token.h>
Public Types | |
enum | TokenFlags { StartOfLine = 0x01, LeadingSpace = 0x02, DisableExpand = 0x04, NeedsCleaning = 0x08, LeadingEmptyMacro = 0x10, HasUDSuffix = 0x20, HasUCN = 0x40, IgnoredComma = 0x80, StringifiedInMacro = 0x100, CommaAfterElided = 0x200 } |
Public Member Functions | |
tok::TokenKind | getKind () const |
void | setKind (tok::TokenKind K) |
bool | is (tok::TokenKind K) const |
is/isNot - Predicates to check if this token is a specific kind, as in "if (Tok.is(tok::l_brace)) {...}". More... | |
bool | isNot (tok::TokenKind K) const |
bool | isOneOf (tok::TokenKind K1, tok::TokenKind K2) const |
template<typename... Ts> | |
bool | isOneOf (tok::TokenKind K1, tok::TokenKind K2, Ts...Ks) const |
bool | isAnyIdentifier () const |
Return true if this is a raw identifier (when lexing in raw mode) or a non-keyword identifier (when lexing in non-raw mode). More... | |
bool | isLiteral () const |
Return true if this is a "literal", like a numeric constant, string, etc. More... | |
bool | isAnnotation () const |
Return true if this is any of tok::annot_* kind tokens. More... | |
SourceLocation | getLocation () const |
Return a source location identifier for the specified offset in the current file. More... | |
unsigned | getLength () const |
void | setLocation (SourceLocation L) |
void | setLength (unsigned Len) |
SourceLocation | getAnnotationEndLoc () const |
void | setAnnotationEndLoc (SourceLocation L) |
SourceLocation | getLastLoc () const |
SourceLocation | getEndLoc () const |
SourceRange | getAnnotationRange () const |
SourceRange of the group of tokens that this annotation token represents. More... | |
void | setAnnotationRange (SourceRange R) |
const char * | getName () const |
void | startToken () |
Reset all flags to cleared. More... | |
IdentifierInfo * | getIdentifierInfo () const |
void | setIdentifierInfo (IdentifierInfo *II) |
const void * | getEofData () const |
void | setEofData (const void *D) |
StringRef | getRawIdentifier () const |
getRawIdentifier - For a raw identifier token (i.e., an identifier lexed in raw mode), returns a reference to the text substring in the buffer if known. More... | |
void | setRawIdentifierData (const char *Ptr) |
const char * | getLiteralData () const |
getLiteralData - For a literal token (numeric constant, string, etc), this returns a pointer to the start of it in the text buffer if known, null otherwise. More... | |
void | setLiteralData (const char *Ptr) |
void * | getAnnotationValue () const |
void | setAnnotationValue (void *val) |
void | setFlag (TokenFlags Flag) |
Set the specified flag. More... | |
bool | getFlag (TokenFlags Flag) const |
Get the specified flag. More... | |
void | clearFlag (TokenFlags Flag) |
Unset the specified flag. More... | |
unsigned | getFlags () const |
Return the internal represtation of the flags. More... | |
void | setFlagValue (TokenFlags Flag, bool Val) |
Set a flag to either true or false. More... | |
bool | isAtStartOfLine () const |
isAtStartOfLine - Return true if this token is at the start of a line. More... | |
bool | hasLeadingSpace () const |
Return true if this token has whitespace before it. More... | |
bool | isExpandDisabled () const |
Return true if this identifier token should never be expanded in the future, due to C99 6.10.3.4p2. More... | |
bool | isObjCAtKeyword (tok::ObjCKeywordKind objcKey) const |
Return true if we have an ObjC keyword identifier. More... | |
tok::ObjCKeywordKind | getObjCKeywordID () const |
Return the ObjC keyword kind. More... | |
bool | needsCleaning () const |
Return true if this token has trigraphs or escaped newlines in it. More... | |
bool | hasLeadingEmptyMacro () const |
Return true if this token has an empty macro before it. More... | |
bool | hasUDSuffix () const |
Return true if this token is a string or character literal which has a ud-suffix. More... | |
bool | hasUCN () const |
Returns true if this token contains a universal character name. More... | |
bool | stringifiedInMacro () const |
Returns true if this token is formed by macro by stringizing or charizing operator. More... | |
bool | commaAfterElided () const |
Returns true if the comma after this token was elided. More... | |
Token - This structure provides full information about a lexed token.
It is not intended to be space efficient, it is intended to return as much information as possible about each returned token. This is expected to be compressed into a smaller form if memory footprint is important.
The parser can create a special "annotation token" representing a stream of tokens that were parsed and semantically resolved, e.g.: "foo::MyClass<int>" can be represented by a single typename annotation token that carries information about the SourceRange of the tokens and the type object.
|
inline |
|
inline |
Returns true if the comma after this token was elided.
Definition at line 300 of file Token.h.
References CommaAfterElided, and getFlag().
|
inline |
Definition at line 137 of file Token.h.
References clang::SourceLocation::getFromRawEncoding(), and isAnnotation().
Referenced by getAnnotationRange(), getEndLoc(), and getLastLoc().
|
inline |
SourceRange of the group of tokens that this annotation token represents.
Definition at line 157 of file Token.h.
References getAnnotationEndLoc(), and getLocation().
Referenced by clang::Parser::TryAnnotateTypeOrScopeToken().
|
inline |
Definition at line 223 of file Token.h.
References isAnnotation().
Referenced by clang::Parser::getTypeAnnotation(), and clang::Parser::ParseTopLevelDecl().
|
inline |
Definition at line 150 of file Token.h.
References getAnnotationEndLoc(), getLength(), getLocation(), clang::SourceLocation::getLocWithOffset(), and isAnnotation().
Referenced by parseDeclareSimdClauses().
|
inline |
Definition at line 189 of file Token.h.
References clang::comments::tok::eof, and is().
|
inline |
Get the specified flag.
Definition at line 238 of file Token.h.
Referenced by commaAfterElided(), hasLeadingEmptyMacro(), hasLeadingSpace(), hasUCN(), hasUDSuffix(), isAtStartOfLine(), isExpandDisabled(), needsCleaning(), and stringifiedInMacro().
|
inline |
Return the internal represtation of the flags.
This is only intended for low-level operations such as writing tokens to disk.
Definition at line 251 of file Token.h.
Referenced by clang::ASTWriter::AddToken().
|
inline |
Definition at line 176 of file Token.h.
References clang::comments::tok::eof, is(), isAnnotation(), isLiteral(), and isNot().
Referenced by clang::Sema::ActOnPragmaUnused(), clang::ASTWriter::AddToken(), clang::format::ContinuationIndenter::addTokenToState(), clang::MacroArgs::ArgNeedsPreexpansion(), clang::TokenConcatenation::AvoidConcat(), clang::Preprocessor::CheckMacroName(), diagnoseAutoModuleImport(), EvaluateDefined(), EvaluateFeatureLikeBuiltinMacro(), EvaluateValue(), ExpectFeatureIdentifierInfo(), GetFirstChar(), getObjCKeywordID(), clang::Lexer::getSpelling(), clang::Preprocessor::getSpelling(), clang::Preprocessor::HandleDirective(), clang::Preprocessor::HandleIdentifier(), clang::Preprocessor::HandlePoisonedIdentifier(), clang::PragmaNamespace::HandlePragma(), clang::format::FormatToken::is(), isConfigurationPattern(), clang::MacroInfo::isIdenticalTo(), isObjCAtKeyword(), isSameToken(), isTrivialSingleTokenExpansion(), isWideStringLiteralFromMacro(), clang::TokenLexer::Lex(), clang::Preprocessor::LexAfterModuleImport(), clang::Preprocessor::LexOnOffSwitch(), clang::Preprocessor::MaybeHandlePoisonedIdentifier(), clang::format::mustBeJSIdent(), clang::TokenValue::operator==(), ParseAlignPragma(), parseDeclareSimdClauses(), clang::Parser::ParseMSAsmIdentifier(), parseOpenMPReductionId(), clang::Parser::ParseUnqualifiedId(), PragmaLoopHintString(), PrintPreprocessedTokens(), clang::Parser::TryAnnotateTypeOrScopeToken(), clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec(), and clang::Preprocessor::TypoCorrectToken().
|
inline |
Definition at line 89 of file Token.h.
Referenced by clang::Sema::ActOnCharacterConstant(), clang::ASTWriter::AddToken(), clang::format::WhitespaceManager::addUntouchableToken(), clang::TokenConcatenation::AvoidConcat(), clang::Preprocessor::CheckMacroName(), clang::Lexer::ComputePreamble(), clang::Preprocessor::DumpToken(), EvaluateDirectiveSubExpr(), EvaluateFeatureLikeBuiltinMacro(), EvaluateHasIncludeCommon(), EvaluateValue(), getBeginningOfFileToken(), clang::format::FormatToken::getPrecedence(), getSpellingSlow(), clang::Preprocessor::HandleDirective(), isAnnotation(), isAnyIdentifier(), isConfigurationPattern(), clang::format::FormatToken::isFunctionLikeKeyword(), clang::MacroInfo::isIdenticalTo(), isLiteral(), clang::Preprocessor::IsPreviousCachedToken(), isSameToken(), clang::format::FormatToken::isSimpleTypeSpecifier(), clang::format::FormatToken::isStringLiteral(), isTypeQualifier(), clang::format::FormatToken::isUnaryOperator(), clang::Preprocessor::LexAfterModuleImport(), clang::Preprocessor::LexNonComment(), clang::Preprocessor::LexUnexpandedNonComment(), clang::TokenValue::operator==(), parseOpenMPReductionId(), ParseReductionId(), clang::Parser::ParseTopLevelDecl(), PrintPreprocessedTokens(), clang::format::WhitespaceManager::replaceWhitespace(), clang::Parser::SkipMalformedDecl(), clang::Parser::SkipUntil(), clang::MacroArgs::StringifyArgument(), and clang::html::SyntaxHighlight().
|
inline |
Definition at line 146 of file Token.h.
References getAnnotationEndLoc(), getLocation(), and isAnnotation().
Referenced by clang::Parser::TryAnnotateTypeOrScopeToken().
|
inline |
Definition at line 126 of file Token.h.
References isAnnotation().
Referenced by clang::Sema::ActOnNumericConstant(), clang::ASTWriter::AddToken(), clang::TokenConcatenation::AvoidConcat(), clang::Preprocessor::ConcatenateIncludeName(), clang::Preprocessor::DumpToken(), getBeginningOfFileToken(), getEndLoc(), GetFirstChar(), GetLineValue(), clang::StringLiteralParser::getOffsetOfStringByte(), getRawIdentifier(), clang::Lexer::getSpelling(), clang::Preprocessor::getSpelling(), clang::Preprocessor::getSpellingOfSingleCharacterNumericConstant(), getSpellingSlow(), clang::TokenLexer::Lex(), clang::Lexer::MeasureTokenLength(), NormalizeLine(), PrintPreprocessedTokens(), clang::RewriteMacrosInInput(), clang::MacroArgs::StringifyArgument(), clang::html::SyntaxHighlight(), updateConsecutiveMacroArgTokens(), and UseNSOptionsMacro().
|
inline |
getLiteralData - For a literal token (numeric constant, string, etc), this returns a pointer to the start of it in the text buffer if known, null otherwise.
Definition at line 214 of file Token.h.
References isLiteral().
Referenced by GetFirstChar(), clang::Lexer::getSpelling(), clang::Preprocessor::getSpellingOfSingleCharacterNumericConstant(), PrintPreprocessedTokens(), and UseNSOptionsMacro().
|
inline |
Return a source location identifier for the specified offset in the current file.
Definition at line 123 of file Token.h.
References clang::SourceLocation::getFromRawEncoding().
Referenced by clang::Sema::ActOnCharacterConstant(), clang::Sema::ActOnNumericConstant(), clang::Sema::ActOnPragmaUnused(), clang::ASTWriter::AddToken(), clang::TokenConcatenation::AvoidConcat(), clang::Preprocessor::CheckEndOfDirective(), clang::Preprocessor::CheckMacroName(), clang::Sema::ClassifyName(), ComputeDATE_TIME(), clang::Lexer::ComputePreamble(), clang::Preprocessor::ConcatenateIncludeName(), clang::Parser::ConsumeToken(), clang::Parser::Diag(), clang::Preprocessor::Diag(), diagnoseUnexpectedOperator(), clang::DoPrintPreprocessedInput(), clang::Preprocessor::DumpToken(), EvaluateDefined(), EvaluateDirectiveSubExpr(), EvaluateFeatureLikeBuiltinMacro(), EvaluateHasIncludeCommon(), EvaluateValue(), clang::Parser::ExitScope(), ExpectFeatureIdentifierInfo(), findDirectives(), FixDigraph(), getAnnotationRange(), getBeginningOfFileToken(), getEndLoc(), GetFirstChar(), getLastLoc(), GetLineValue(), clang::StringLiteralParser::getOffsetOfStringByte(), clang::Lexer::getSpelling(), clang::Preprocessor::getSpellingOfSingleCharacterNumericConstant(), clang::Preprocessor::HandleDirective(), clang::Preprocessor::HandleEndOfFile(), clang::Preprocessor::HandleIdentifier(), clang::Preprocessor::HandlePragmaDependency(), clang::Preprocessor::HandlePragmaIncludeAlias(), clang::Preprocessor::HandlePragmaPopMacro(), clang::Preprocessor::HandlePragmaSystemHeader(), clang::html::HighlightMacros(), clang::TokenLexer::Init(), clang::Preprocessor::IsPreviousCachedToken(), clang::TokenLexer::Lex(), clang::Preprocessor::LexAfterModuleImport(), clang::PreprocessorLexer::LexIncludeFilename(), NormalizeLine(), ParseAlignPragma(), parseDeclareSimdClauses(), ParseLoopHintValue(), clang::Parser::ParseMSAsmIdentifier(), clang::Parser::ParseOpenMPParensExpr(), parseOpenMPReductionId(), clang::Parser::ParseOpenMPVarList(), clang::Preprocessor::ParsePragmaPushOrPopMacro(), clang::Preprocessor::parseSimpleIntegerLiteral(), clang::Parser::ParseTopLevelDecl(), clang::Parser::ParseUnqualifiedId(), PrintPreprocessedTokens(), ReadLineMarkerFlags(), clang::RewriteMacrosInInput(), clang::arcmt::trans::MigrationContext::rewritePropertyAttribute(), clang::html::SyntaxHighlight(), clang::Parser::TryAnnotateTypeOrScopeToken(), clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec(), clang::Parser::TryConsumeToken(), and updateConsecutiveMacroArgTokens().
|
inline |
Definition at line 165 of file Token.h.
References clang::tok::getTokenName().
tok::ObjCKeywordKind Token::getObjCKeywordID | ( | ) | const |
Return the ObjC keyword kind.
getObjCKeywordID - Return the ObjC keyword kind.
Definition at line 43 of file Lexer.cpp.
References getIdentifierInfo(), and clang::IdentifierInfo::getObjCKeywordID().
|
inline |
getRawIdentifier - For a raw identifier token (i.e., an identifier lexed in raw mode), returns a reference to the text substring in the buffer if known.
Definition at line 202 of file Token.h.
References getLength(), and is().
Referenced by clang::Lexer::ComputePreamble(), clang::Lexer::getSpelling(), and clang::Preprocessor::LookUpIdentifierInfo().
|
inline |
Return true if this token has an empty macro before it.
Definition at line 286 of file Token.h.
References getFlag(), and LeadingEmptyMacro.
|
inline |
Return true if this token has whitespace before it.
Definition at line 269 of file Token.h.
References getFlag(), and LeadingSpace.
Referenced by buildMSAsmString(), clang::Preprocessor::ConcatenateIncludeName(), clang::Preprocessor::DumpToken(), clang::html::HighlightMacros(), clang::TokenLexer::Init(), clang::MacroInfo::isIdenticalTo(), PrintPreprocessedTokens(), clang::RewriteMacrosInInput(), and clang::MacroArgs::StringifyArgument().
|
inline |
Returns true if this token contains a universal character name.
Definition at line 293 of file Token.h.
References getFlag(), and HasUCN.
Referenced by clang::Lexer::getSpelling(), clang::Preprocessor::getSpelling(), and clang::Preprocessor::LookUpIdentifierInfo().
|
inline |
Return true if this token is a string or character literal which has a ud-suffix.
Definition at line 290 of file Token.h.
References getFlag(), and HasUDSuffix.
Referenced by clang::TokenConcatenation::AvoidConcat(), EvaluateValue(), clang::Preprocessor::FinishLexStringLiteral(), and clang::Preprocessor::ParsePragmaPushOrPopMacro().
|
inline |
is/isNot - Predicates to check if this token is a specific kind, as in "if (Tok.is(tok::l_brace)) {...}".
Definition at line 94 of file Token.h.
Referenced by clang::TokenConcatenation::AvoidConcat(), buildMSAsmString(), clang::Preprocessor::CheckEndOfDirective(), clang::Preprocessor::CheckMacroName(), clang::Sema::ClassifyName(), clang::Preprocessor::ConcatenateIncludeName(), clang::BalancedDelimiterTracker::consumeClose(), clang::BalancedDelimiterTracker::consumeOpen(), clang::Preprocessor::CreateString(), diagnoseUnexpectedOperator(), clang::DoPrintPreprocessedInput(), EvaluateDefined(), EvaluateHasIncludeCommon(), EvaluateValue(), findDirectives(), clang::Preprocessor::FinishLexStringLiteral(), clang::PTHLexer::getEOF(), getEofData(), getIdentifierInfo(), getRawIdentifier(), clang::Lexer::getSpelling(), clang::Preprocessor::getSpellingOfSingleCharacterNumericConstant(), clang::MacroArgs::getUnexpArgument(), clang::Preprocessor::HandleDirective(), clang::Preprocessor::HandleMicrosoftCommentPaste(), clang::Preprocessor::HandlePragmaDependency(), clang::Preprocessor::HandlePragmaIncludeAlias(), clang::Preprocessor::HandlePragmaPoison(), clang::html::HighlightMacros(), clang::format::FormatToken::is(), IsCommonTypo(), isGCCAsmStatement(), clang::TokenLexer::isNextTokenLParen(), isOneOf(), clang::TokenLexer::isParsingPreprocessorDirective(), isResultTypeOrTemplate(), isWideStringLiteralFromMacro(), clang::TokenLexer::Lex(), clang::Preprocessor::Lex(), clang::PreprocessorLexer::LexIncludeFilename(), LexRawTokensFromMainFile(), clang::Parser::ParseAssignmentExpression(), parseDeclareSimdClauses(), ParseLoopHintValue(), clang::Parser::ParseMSAsmIdentifier(), parseOpenMPReductionId(), clang::Parser::ParseOpenMPVarList(), clang::Preprocessor::parseSimpleIntegerLiteral(), clang::Parser::ParseTopLevelDecl(), clang::Parser::ParseUnqualifiedId(), PrintPreprocessedTokens(), ReadLineMarkerFlags(), clang::Lexer::ReadToEndOfLine(), clang::RewriteMacrosInInput(), clang::arcmt::trans::MigrationContext::rewritePropertyAttribute(), setEofData(), setRawIdentifierData(), clang::Parser::SkipMalformedDecl(), clang::Parser::SkipUntil(), clang::MacroArgs::StringifyArgument(), clang::Parser::TryAnnotateCXXScopeToken(), clang::Parser::TryAnnotateTypeOrScopeToken(), and clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec().
|
inline |
Return true if this is any of tok::annot_* kind tokens.
Definition at line 117 of file Token.h.
References getKind(), and clang::tok::isAnnotation().
Referenced by clang::Preprocessor::AnnotateCachedTokens(), clang::TokenConcatenation::AvoidConcat(), ExpectFeatureIdentifierInfo(), getAnnotationEndLoc(), getAnnotationValue(), getEndLoc(), getIdentifierInfo(), getLastLoc(), getLength(), clang::TokenLexer::Lex(), ParseOpenMPDirectiveKind(), clang::Preprocessor::ReplaceLastTokenWithAnnotation(), setAnnotationEndLoc(), setAnnotationValue(), setLength(), and clang::Parser::TryAnnotateTypeOrScopeToken().
|
inline |
Return true if this is a raw identifier (when lexing in raw mode) or a non-keyword identifier (when lexing in non-raw mode).
Definition at line 106 of file Token.h.
References getKind(), and clang::tok::isAnyIdentifier().
|
inline |
isAtStartOfLine - Return true if this token is at the start of a line.
Definition at line 265 of file Token.h.
References getFlag(), and StartOfLine.
Referenced by buildMSAsmString(), clang::Lexer::ComputePreamble(), clang::Preprocessor::DumpToken(), clang::html::HighlightMacros(), clang::TokenLexer::Init(), clang::MacroInfo::isIdenticalTo(), clang::PTHLexer::Lex(), NormalizeLine(), PrintPreprocessedTokens(), clang::RewriteMacrosInInput(), clang::Parser::SkipMalformedDecl(), clang::MacroArgs::StringifyArgument(), and clang::html::SyntaxHighlight().
|
inline |
Return true if this identifier token should never be expanded in the future, due to C99 6.10.3.4p2.
Definition at line 273 of file Token.h.
References DisableExpand, and getFlag().
Referenced by clang::Preprocessor::DumpToken(), and clang::Preprocessor::HandleIdentifier().
|
inline |
Return true if this is a "literal", like a numeric constant, string, etc.
Definition at line 112 of file Token.h.
References getKind(), and clang::tok::isLiteral().
Referenced by clang::Preprocessor::CreateString(), GetFirstChar(), getIdentifierInfo(), getLiteralData(), clang::Lexer::getSpelling(), isWideStringLiteralFromMacro(), clang::PTHLexer::Lex(), clang::format::mustBeJSIdentOrValue(), clang::format::ContinuationIndenter::mustBreak(), PrintPreprocessedTokens(), setLiteralData(), and UseNSOptionsMacro().
|
inline |
Definition at line 95 of file Token.h.
Referenced by clang::MacroArgs::ArgNeedsPreexpansion(), clang::TokenConcatenation::AvoidConcat(), clang::CacheTokens(), clang::Preprocessor::CheckEndOfDirective(), clang::Preprocessor::ConcatenateIncludeName(), clang::Preprocessor::DiscardUntilEndOfDirective(), DoPrintMacros(), EvaluateDefined(), EvaluateDirectiveSubExpr(), EvaluateFeatureLikeBuiltinMacro(), EvaluateHasIncludeCommon(), EvaluateValue(), clang::DumpRawTokensAction::ExecuteAction(), clang::DumpTokensAction::ExecuteAction(), clang::PreprocessOnlyAction::ExecuteAction(), findDirectives(), clang::Preprocessor::FinishLexStringLiteral(), clang::MacroArgs::getArgLength(), getIdentifierInfo(), GetLineValue(), clang::Preprocessor::getSpelling(), clang::Preprocessor::HandleMicrosoftCommentPaste(), clang::Preprocessor::HandlePragmaDependency(), clang::Preprocessor::HandlePragmaIncludeAlias(), clang::Preprocessor::HandlePragmaPoison(), clang::html::HighlightMacros(), clang::Preprocessor::LexOnOffSwitch(), LexRawTokensFromMainFile(), clang::Parser::ConditionDeclarationOrInitStatementState::markNotExpression(), ParseAlignPragma(), parseDeclareSimdClauses(), ParseLoopHintValue(), clang::Parser::ParseMSAsmIdentifier(), clang::Parser::ParseOpenMPVarList(), clang::Preprocessor::ParsePragmaPushOrPopMacro(), clang::Parser::ParseUnqualifiedId(), clang::RewriteIncludesInInput(), clang::RewriteMacrosInInput(), clang::arcmt::trans::MigrationContext::rewritePropertyAttribute(), clang::Parser::SkipUntil(), clang::MacroArgs::StringifyArgument(), clang::html::SyntaxHighlight(), clang::format::tokenCanStartNewLine(), clang::TokenRewriter::TokenRewriter(), and clang::Parser::TryConsumeToken().
bool Token::isObjCAtKeyword | ( | tok::ObjCKeywordKind | objcKey | ) | const |
Return true if we have an ObjC keyword identifier.
isObjCAtKeyword - Return true if we have an ObjC keyword identifier.
Definition at line 36 of file Lexer.cpp.
References getIdentifierInfo().
Referenced by clang::format::FormatToken::isObjCAtKeyword(), and clang::Parser::SkipMalformedDecl().
|
inline |
Definition at line 96 of file Token.h.
References is().
Referenced by clang::TokenConcatenation::AvoidConcat(), clang::Sema::ClassifyName(), EvaluateFeatureLikeBuiltinMacro(), isConfigurationPattern(), isEndOfTemplateArgument(), isOneOf(), isValidAfterIdentifierInDeclarator(), clang::Parser::ParseOpenMPVarList(), and clang::Parser::SkipMalformedDecl().
|
inline |
|
inline |
Return true if this token has trigraphs or escaped newlines in it.
Definition at line 282 of file Token.h.
References getFlag(), and NeedsCleaning.
Referenced by clang::Lexer::ComputePreamble(), clang::Preprocessor::DumpToken(), GetFirstChar(), clang::Lexer::getSpelling(), clang::Preprocessor::getSpelling(), clang::Preprocessor::getSpellingOfSingleCharacterNumericConstant(), getSpellingSlow(), clang::Preprocessor::LookUpIdentifierInfo(), and PrintPreprocessedTokens().
|
inline |
Definition at line 141 of file Token.h.
References clang::SourceLocation::getRawEncoding(), and isAnnotation().
Referenced by EnterAnnotationToken(), clang::Preprocessor::HandleEndOfFile(), setAnnotationRange(), clang::Parser::TryAnnotateTypeOrScopeToken(), and clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec().
|
inline |
Definition at line 160 of file Token.h.
References clang::SourceRange::getBegin(), clang::SourceRange::getEnd(), setAnnotationEndLoc(), and setLocation().
|
inline |
Definition at line 227 of file Token.h.
References isAnnotation().
Referenced by EnterAnnotationToken(), and clang::Preprocessor::HandleEndOfFile().
|
inline |
Definition at line 193 of file Token.h.
References clang::comments::tok::eof, and is().
|
inline |
Set the specified flag.
Definition at line 233 of file Token.h.
Referenced by clang::Preprocessor::HandleIdentifier(), clang::PTHLexer::Lex(), clang::TokenLexer::Lex(), clang::ASTReader::ReadToken(), and setFlagValue().
|
inline |
Set a flag to either true or false.
Definition at line 256 of file Token.h.
References clearFlag(), and setFlag().
Referenced by clang::TokenLexer::Lex().
|
inline |
Definition at line 185 of file Token.h.
Referenced by clang::Sema::ActOnIdExpression(), clang::Preprocessor::CheckMacroName(), EvaluateHasIncludeCommon(), clang::Preprocessor::HandleIdentifier(), clang::PTHLexer::Lex(), clang::Preprocessor::LookUpIdentifierInfo(), clang::ASTReader::ReadToken(), and clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec().
|
inline |
Definition at line 90 of file Token.h.
Referenced by clang::Sema::ActOnIdExpression(), clang::TokenRewriter::AddTokenBefore(), EnterAnnotationToken(), EvaluateFeatureLikeBuiltinMacro(), EvaluateHasIncludeCommon(), findDirectives(), FixDigraph(), GenerateNewArgTokens(), clang::Preprocessor::HandleIdentifier(), clang::html::HighlightMacros(), clang::PTHLexer::Lex(), clang::TokenLexer::Lex(), clang::Preprocessor::LookUpIdentifierInfo(), ParseLoopHintValue(), clang::Parser::ParseMSAsmIdentifier(), clang::Parser::Parser(), clang::ASTReader::ReadToken(), clang::MacroArgs::StringifyArgument(), clang::Parser::TryAnnotateTypeOrScopeToken(), and clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec().
|
inline |
Definition at line 132 of file Token.h.
References isAnnotation().
Referenced by clang::TokenRewriter::AddTokenBefore(), clang::Preprocessor::CreateString(), FixDigraph(), GenerateNewArgTokens(), clang::PTHLexer::Lex(), and clang::ASTReader::ReadToken().
|
inline |
Definition at line 218 of file Token.h.
References isLiteral().
Referenced by clang::Preprocessor::CreateString(), and clang::PTHLexer::Lex().
|
inline |
Definition at line 131 of file Token.h.
References clang::SourceLocation::getRawEncoding().
Referenced by clang::Sema::ActOnIdExpression(), clang::TokenRewriter::AddTokenBefore(), clang::Preprocessor::CreateString(), EnterAnnotationToken(), FixDigraph(), GenerateNewArgTokens(), clang::Preprocessor::HandleEndOfFile(), clang::PTHLexer::Lex(), clang::TokenLexer::Lex(), ParseLoopHintValue(), clang::ASTReader::ReadToken(), setAnnotationRange(), clang::Parser::TryAnnotateTypeOrScopeToken(), clang::Parser::TryAnnotateTypeOrScopeTokenAfterScopeSpec(), and updateConsecutiveMacroArgTokens().
|
inline |
Definition at line 206 of file Token.h.
References is().
Referenced by clang::Preprocessor::CreateString().
|
inline |
Reset all flags to cleared.
Definition at line 168 of file Token.h.
References clang::SourceLocation::getRawEncoding().
Referenced by clang::Sema::ActOnIdExpression(), clang::TokenRewriter::AddTokenBefore(), ComputeDATE_TIME(), EnterAnnotationToken(), GenerateNewArgTokens(), clang::Preprocessor::HandleEndOfFile(), clang::PTHLexer::Lex(), clang::TokenLexer::Lex(), ParseLoopHintValue(), clang::Parser::ParseMSAsmIdentifier(), clang::Preprocessor::ParsePragmaPushOrPopMacro(), clang::Parser::Parser(), PrintPreprocessedTokens(), clang::ASTReader::ReadToken(), clang::arcmt::trans::MigrationContext::rewritePropertyAttribute(), and clang::MacroArgs::StringifyArgument().
|
inline |
Returns true if this token is formed by macro by stringizing or charizing operator.
Definition at line 297 of file Token.h.
References getFlag(), and StringifiedInMacro.
Referenced by isWideStringLiteralFromMacro().