Add and require version_added lexer attribute#2589
Merged
Conversation
version_added lexer attributeversion_added lexer attribute
a1f7daf to
452dc48
Compare
Contributor
Author
|
Today I learnt that GitHub doesn't test the PR branch but the result of merging that branch with |
452dc48 to
4f35a30
Compare
Contributor
Author
|
Done. For the record, here's the script I've used for the third commit:import importlib
import inspect
from pathlib import Path
import re
from pygments.lexers import LEXERS
for class_name, (module_name, _, _, _, _) in LEXERS.items():
mod = importlib.import_module(module_name)
mod = importlib.reload(mod)
cls = getattr(mod, class_name)
file = inspect.getsourcefile(cls)
lines, first_line_index = inspect.getsourcelines(cls)
first_line_index -= 1 # make it zero-based
end_line_index = first_line_index + len(lines)
docstring_finished = False
for version_added_index, version_added_line in enumerate(lines):
if m := re.match(r" .. versionadded:: (.*)", version_added_line):
version_added = m.group(1)
cursor = version_added_index
del lines[version_added_index]
cursor -= 1
if not lines[version_added_index-1].strip():
del lines[version_added_index-1]
cursor -= 1
break
else:
version_added = ''
assert lines[1].startswith(' """'), f"{file}:{first_line_index}"
if lines[1].endswith('"""\n'):
docstring_finished = True
cursor = 2
if not docstring_finished:
for i, line in enumerate(lines[cursor:], cursor):
if line == ' """\n':
cursor = i+1
break
else:
assert False, f"{file}:{first_line_index}"
unlocked = False
for insertion_index, line in enumerate(lines[cursor:], cursor):
if re.match(" (name|url|aliases|filenames|mimetypes) ?=", line):
unlocked = True
elif unlocked and not line.startswith(" ") and not re.match(r" (\[|\])", line):
break
else:
insertion_index += 1
new_version_added_line = f" version_added = {version_added!r}\n"
lines.insert(insertion_index, new_version_added_line)
all_lines = Path(file).read_text(encoding="utf-8").splitlines(keepends=True)
all_lines[first_line_index:end_line_index] = lines
Path(file).write_text(''.join(all_lines), encoding="utf-8") |
Member
|
Very nice! |
Contributor
Author
|
Should we merge this one? @Anteru What do you think about it? |
This was inactive due to the misspelling as "mimetimes", and we don't want it because it conflicts with XML.
That way, we can set it to "" for old lexers, and check that it's present on new lexers. (In the future, we might also use it for better presentation in the documentation.)
4f35a30 to
40bfc41
Compare
Contributor
Author
|
(Updated again for new lexers in |
Collaborator
|
Looks good to me, thanks! I assume finding the ones which are set to |
Anteru
approved these changes
Nov 26, 2023
Contributor
Author
|
Yes, that's it. It's not always easy to find the version for old lexers; some of them have been moved around, and there are also some quirks in the Git history like commit db1e5ef (I think Pygments was originally using Mercurial and migrated to Git at some point?). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Next step for #2585 :)
MXML: Remove bogus mimetypes line
Savi: Fix indentation
Move versionadded data to a lexer attribute
That way, we can set it to "" for old lexers, and check that it's
present on new lexers. (In the future, we might also use it for better
presentation in the documentation.)
Make documentation use the new version_added lexer attribute
Require the version_added lexer attribute
Harmonize version_added format