1. fix a performance regression when using line-by-line highlighting
* the root cause is that chroma's `lexers.Get` is slow and a lexer cache
is missing during recent changes
2. clarify the chroma lexer detection behavior
* now we fully manage our logic to detect lexer, and handle overriding
problems, everything is fully under control
3. clarify "code analyze" behavior, now only 2 usages:
* only use file name and language to detect lexer (very fast), mainly
for "diff" page which contains a lot of files
* if no lexer is detected by file name and language, use code content to
detect again (slow), mainly for "view file" or "blame" page, which can
get best result
4. fix git diff bug, it caused "broken pipe" error for large diff files
* Fix#35252
* Fix#35999
* Improve diff rendering, don't add unnecessary "added"/"removed" tags for a full-line change
* Also fix a "space trimming" bug in #36539 and add tests
* Use chroma "SQL" lexer instead of "MySQL" to workaround a bug (35999)
For when Enry correctly recognized the language, but returns the
language name in a way that isn't recognized by chroma.
Resolves https://github.com/go-gitea/gitea/issues/22443
---------
Co-authored-by: Moritz Jörg <moritz.jorg@oceanbox.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
1. Full file highlighting (fix the legacy todo "we should instead
highlight the whole file at once")
* Fix#24383
2. Correctly covert file content encoding
3. Remove dead code, split large for-loop into small functions/blocks to
make code maintainable
Change all license headers to comply with REUSE specification.
Fix#16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
* Fix copy/paste of empty newlines again
Fixes: https://github.com/go-gitea/gitea/issues/19331
Regressed by: https://github.com/go-gitea/gitea/pull/18270
Needed to do another newline addition to the Chroma output HTML to get
copy/paste work again. The previous replacement conditions are probably
obsolete, but as I'm not 100% sure, I opted to keep them.
Specifically, the Chroma HTML change mentioned in
https://github.com/go-gitea/gitea/pull/18270#issuecomment-1013350246
broke our previous newline replacement for such empty lines.
Also included are a few changes to make the test more pleasant to work
with.
* run go mod tidy
* add util.Dedent
* copy in the code
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Use check attribute code to check the assigned language of a file and send that in to
chroma as a hint for the language of the file.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Ensure empty lines are copiable and final new line too
When files are highlighted the newline character needs to be added in a whitespace
compliant mode. Also ensure the final empty newline is rendered.
Fix#16434
* Add test and ensure spans closed
Signed-off-by: Andrew Thornton <art27@cantab.net>
