It is finally time to commit the entire syntax highlighting code and first is always the pattern language #115
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… identifier types of variable definitions only. Fixed typo in parser_manager.cpp. Parser errors were overwriting preprocessor/lexer errors. Make sure they are all added. If ast fails to be created make sure that the parsed tokens are saved before returning. Change to a more appropriate name in preprocessor and remove excluded location validation. Rearrange identifier type enumeration values to support value ordering of importance (bigger are more important types and preferred if choice can be made). Also removed types that were not identifiers.
…here the next break will occur. The value returned by getPauseLine() is the number of the line above the one it wants, but the values it may return range from zero to lines-1. When the code that's being executed cannot be seen because it is in another file, the debugger sets the line to zero which in turn is adjusted to line number by subtracting one. As negative line number locations are asserted against getPauseLine() now makes sure that the smallest number it can return is one. fix: found a bug where multi variable definitions were being label incorrectly by the parser which caused the highlighter to ignore all instances of the variables. Due to the name of the function involved it appeared that only custom type member definitions were being processed, but it appears the function variable definitions were processed by the code. In order to tell one set of variables from the other an specific node variable that only exists in one of the two provides the litmus test.
…preprocessor_improvements
…preprocessor_improvements # Conflicts: # lib/source/pl/core/evaluator.cpp
…first is always the pattern language. there are also a number of fixes and improvements as listed next. fix: problem with having no spaces between ifdef nad endif directives. renamed patternLocalvariable to localVariable and patternPlacedVariable to calculatedPointer Also introduced a view type so now we have 4 struct types, 3 function types and 2 global types. fix: global arrays with new initialization syntax showed as unknown types. Syntax highlighting doesn't use the ast anymore. fix: preprocessor directives cannot have code on the same line they are except for comments. Any extra tokens located after a fully parsed directive are simply ignored. in earlier versions the highlighting code used to sets of tokens. It needed to have access to the token sequence as it is before preprocessingso tthat it contains all comments and directives. It also needed access to the sequence that was created by the parser because it contains all the identifier definitions types. Now the rwo are merged together in the preprocessor making highlighting simpler
… were not being labeled properly as arrays or plain variables. fix: The end of program token was causing iterator out of bounds issues because it was assigned the default size of 1 even though it doesn't add to the size of the input so its size should reallybe zero. Note: The changes included in this PR are necessary for my ImHex open PR changes to compile, so tests wont be passed on the ImHex PR unless this or parts of it are merged and the pattern language submodule oh ImHex is updated
…r variables. These changes should take care of them all.
…t increasing the number of lines so all subsequent tokens had the wrong line location. This was causing heap corruption errors in some cases when it should be causing index out of bounds errors like in the other cases which is kind of suspicious.
…e argument in its last position then the resulting double closing template delimiters were being parsed as a binary shift operator. The fix extends the inclusion of the `inTemplate` bool to the shift operator so that it can return before adding the wrong result to the mathematical expression.
…ogram token gets removed from token sequence when the directive or comment are removed which results in "unexpected end of file reached" error. Fixed making sure the token gets added if it is missing during token sequence validation prior to returning it.
…code if file was imported could not be made to work. Using the addDefine in Preprocessor never worked so an addDefine was added in PatternLanguage to store system defines but that would add the defines to all the preprocessors and not just the imported ones. The fix consists on creating a new variable in Preprocessor to store defines that come from Preprocessor::addDefine which are not deleted on first run and inserted into the variable that stores the defines on every run. improv: turned function return values and arguments into const references to avoid copies
…ve, the code would silently ignore the include directive. fix: In order to determine if a namespace is valid to correctly highlight it, the preprocessors compile lists of encountered namespaces that the text highlighter can use when testing newly found identifiers. This works fine as long as the preprocessors use a valid way to determine if an identifier is a namespace or not and they weren't.
… namespaces created within imported files. That uses a function and the function had the same incorrect test for namespace identification. Once the function was updated then it gets used everywhere to avoid code duplication.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are also a number of fixes and improvements as listed next.
fix: problem with having no spaces between
ifdefandendifdirectives. renamedpatternLocalvariabletolocalVariableandpatternPlacedVariabletocalculatedPointerAlso introduced a view type so now we have 4 struct types, 3 function types and 2 global types.fix: global arrays with new initialization syntax showed as unknown types.
Syntax highlighting doesn't use the
astanymore.fix: preprocessor directives cannot have code on the same line they are except for comments. Any extra tokens located after a fully parsed directive are simply ignored. Earlier versions of the highlighting code used two sets of tokens. It needed to have access to the token sequence as it is before preprocessing so that it contains all comments and directives. It also needed access to the sequence that was created by the parser because it contains all the identifier definitions types. Now the two are merged together in the preprocessor making highlighting simpler