Problem:
`vim.diagnostic.set()` doesn't actually accept a list of
`vim.Diagnostic` as internally `vim.diagnostic.set()` normalizes the
diagnostics and this normalization is assumed throughout the module.
Solution:
- Add a new type `vim.Diagnostic.Set` which is the input to `vim.diagnostic.set()`
- `col` is now an optional field and defaults to `0` to be consistent
with `vim.diagnostic.match()`.
- Change `table.insert(t, x)` to `table[#table + 1] = x` for improved
type checking.
Problem: `TSLangInfo` annotation does not reflect the structure returned
by `vim.treesitter.language.inspect()`.
Solution: Move version information under new (optional since ABI 15 only)
`TSLangMetadata` field.
**Problem:** `LanguageTree:contains()` considers any range within the
start of the first tree and end of the last tree as "within" the
language tree. In the case of combined injections, this is problematic
because we only want to consider ranges within any of the combined trees
as "contained" (as opposed to any range within the entire range spanned
by all combined trees).
**Solution:** Use a more discriminative check in
`LanguageTree:contains()`.
Problem: conceal_lines cache is invalidated in `on_buf`
which is too late for code calculating text height after a
buffer change but before a redraw (like `lsp/util.lua`).
Solution: Replace `on_buf` with `on_bytes` handler that invalidates
the cache and clears the marks.
This commit will eliminate flicker while editing one open window. It
works by querying previously calculated trees for highlight marks, in
the case that we are still asynchronously parsing in `on_win`.
It will not fully solve the case of flicker when there are multiple open
windows, since the parser will drop previously parsed injection trees
whenever it receives a call to parse a different region of the buffer.
This will require a refactor of `languagetree.lua`.
Problem: Spliced conceal_lines marks after changing the buffer text are
left valid, concealing lines that shouldn't be.
Solution: Set the `invalidate` extmark property.
Problem:
The check for concealing paths in TOCs in the qf syntax file fails
because the TOC tile has changed.
Solution:
Force the qf syntax file to be reloaded after the qf_toc variable
has been set, so that the it can apply the correct settings.
Using the explicit qf_toc key, already used in the syntax file, instead
of the title is less prone to breaking.
It was also already being set for man pages but it had no effect because
the syntax file had already been loaded when the variable was set.
Fixes#33733
Problem:
treesitter injected language ranges sometimes cross over the capture
boundaries when `@combined`.
Solution:
Clip child regions to not spill out of parent regions within
languagetree.lua, and only apply highlights within those regions in
highlighter.lua.
Co-authored-by: Cormac Relf <web@cormacrelf.net>
Remove the `set_timeout` functions for `TSParser` and instead add a timeout
parameter to the regular parse function. Remove these deprecated tree-sitter
API functions and replace them with the preferred `TSParseOptions` style.
Simplify the logic for retrieving the injection ranges for the language
tree. The trees are now also sorted by starting position, regardless of
whether they are part of a combined injection or not. This would be
helpful if ranges are ever to be stored in an interval tree or other
kind of sorted tree structure.
This matches the `iter_captures` functionality to the `iter_matches`
functionality, allowing it to specify a match limit and start depth for
the query iterator.
Problem: Cannot disable individual captures and patterns in treesitter queries.
Solution:
* Expose the corresponding tree-sitter API functions for `TSQuery` object.
* Add documentation for `TSQuery`.
* Return the pattern ID from `get_captures_at_pos()` (and hence `:Inspect!`).
Problem:
:checkhealth vim.treesitter sorts parser entries solely by path,
splitting duplicates and reducing clarity.
Solution:
Sort entries first by name, then by path, so that duplicates are grouped.
**Problem:** Some weird languages have different comment syntax
depending on the location in the code, and we do not have a way to
determine the correct `commentstring` for these special cases.
**Solution:** Allow queries to specify `commentstring` values in
metadata, allowing users/`nvim-treesitter` to provide a better
commenting experience without hugely increasing the scope of the code in
core.
Problem: No way to check the version of a treesitter parser.
Solution: Add version metadata (ABI 15 parsers only) as well as parser state count and supertype information (ABI 15) in `vim.treesitter.language.inspect()`. Also graduate the `abi_version` field, as this is now the official upstream name.
---------
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
Problem: _on_conceal_line callbacks are not invoked if callback has not
let Nvim know it wants to receive them. But this may change on
factors other than what is currently checked (changed buffer).
Solution: Forego this optimization, callback is still guarded behind
'conceallevel'.
TSHighlighter now places marks for conceal_lines metadata. A new
internal decor provider callback _on_conceal_line was added that
instructs the highlighter to place conceal_lines marks whenever the
editor needs to know whether a line is concealed. The bundled markdown
queries use conceal_lines metadata to conceal code block fence lines.
Lua coroutines can yield across non-coroutine function boundaries,
meaning that we don't need to wrap each helper function in a coroutine
and resume it within `_parse()`. If we just have them yield when
appropriate, this will be caught by the top level `_parse()` coroutine,
and resuming the `_parse()` will resume from the position in the helper
function where we yielded last.
Problem: It's difficult to navigate large structured text files (vim
help, checkhealth, Markdown).
Solution: Support `gO` for table of contents and `]]`/`[[` for moving
between headings for all these filetypes using treesitter queries.
Refactor: colorization of highlight groups is moved to the `help` ftplugin
while headings-related functionality is implemented in a private
`vim.treesitter` module for possible future use for other filetypes.
**Problem:** Currently, parsing is asynchronous, but it involves a
(sometimes lengthy) step which finds all injection ranges for a tree by
iterating over that language's injection queries. This causes edits in
large files to be extremely slow, and also causes a long stutter during
the initial parse of a large file.
**Solution:** Break up the injection query iteration over multiple event
loop iterations.
**Problem:** An erroneous query in the treesitter highlighter gives a
deluge of errors that makes the editor almost unusable.
**Solution:** Detach the highlighter after an error is detected, so that
it only gets displayed once (per highlighter instance).
Problem: Computing fold levels for an empty buffer (somehow) breaks the
parser state, resulting in a broken highlighter and foldexpr.
Cached foldexpr parser is invalid after filetype has changed.
Solution: Avoid computing fold levels for empty buffer.
Clear cached foldinfos upon `FileType`.
We need to add a separate variable to keep track of this information,
since we cannot read the length of the valid regions table itself, since
it has holes.
Now `:InspectTree` will show missing nodes as e.g. `(MISSING identifier)`
or `(MISSING ";")` rather than just `(identifier)` or `";"`. This is
doable because the `MISSING` keyword is now valid query syntax.
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
When given, only that range will be checked for validity rather than the
entire tree. This is used in the highlighter to save CPU cycles since we
only need to parse a certain region at a time anyway.
This means that all work previously done by a `_parse()` iteration will
be kept in future iterations. This prevents it from running indefinitely
in some cases where the file is very large and there are 2+ injections.
Problem:
When running an initial parse, parse() returns an empty table rather
than an actual range. In `languagetree.lua`, we manually check if
a parse was incremental to determine the changed parse region.
Solution:
- Always return a range (in the C side) from parse().
- Simplify the language tree code a bit.
- Logger no longer shows empty ranges on the initial parse.
This simplifies some logic in `languagetree.lua`, removing the need for
`_has_regions`, and removing side effects in `:included_regions()`.
Before:
- Edit is made which sets `_regions = nil`
- Upon the next call to `included_regions()` (usually right after we
marked `_regions` as `nil` due to an `_iter_regions()` call), if
`_regions` is nil, we repopulate the table (as long as the tree
actually has regions)
After:
- Edit is made which resets `_regions` if it exists
- `included_regions()` no longer needs to perform this logic itself, and
also no longer needs to read a `_has_regions` variable
Problem: `TSHighlighter:destroy()` causes double-processing of the modeline
and failure of `b:undo_ftplugin`.
Solution: Disable modeline in `TSHighlighter:destroy()` by setting `modeline=false`
if executing `syntaxset` autocommands for the `FileType` event.
Co-authored-by: Daniel Petrovic <daniel.petrovic@ebcont.com>
