Problem: There is no check for buffer validity before processing
semantic tokens response. This can lead to `Invalid buffer id` error
if processing request takes a long time and the buffer is wiped out.
For example, this can happen after by accident navigating to a buffer
from different project which leads to first loading project's
workspace and *then* processing semantic tokens. During that time
a buffer can be wiped out, as navigation to it was by accident.
Solution: Add extra check for buffer validity before processing semantic
tokens response.
* deprecate old signatures
* move to new str_byteindex/str_utfindex signature
* use single-underscore name (double-underscore is reserved for Lua itself)
Problem:
- `vim.highlight` module does not follow `:help dev-name-common`, which
documents the name for "highlight" as "hl".
- Shorter names are usually preferred.
Solution:
Rename `vim.highlight` to `vim.hl`.
This is not a breaking change until 2.0 (or maybe never).
Problem:
str_byteindex_enc could return an error if the index was longer than the
lline length. This was handled in each of the calls to it individually
Solution:
* Fix the call at the source level so that if the index is higher than
the line length, line length is returned as per LSP specification
* Remove pcalls on str_byteindex_enc calls. No longer needed now that
str_byteindex_enc has a bounds check.
This reduces the number of nil checks around buf_versions usage
Test changes were lifted from 5c33815
Co-authored-by: Mathias Fussenegger <f.mathias@zignar.net>
* Revert "fix(lsp): account for changedtick version gap on modified reset (#29170)"
This reverts commit 2e6d295f79.
* Revert "refactor(lsp): replace util.buf_versions with changedtick (#28943)"
This reverts commit 5c33815448.
LspDetach is now triggered by the main on_detach callback that is added
when an LSP client is attached to a buffer. The semantic_tokens module
already includes a LspDetach handler that does the right thing. When the
LspDetach trigger was added to the main LSP on_detach, it created a race
condition in semantic tokens when a buffer was deleted that would
trigger both its own on_detach and the LspDetach handlers. If the former
came last, an error was thrown trying to delete a non-existent augroup
(destroy() was being called twice).
`lsp.util.buf_versions` was already derived from changedtick (`on_lines`
from `buf_attach` synced the version)
As far as I can tell there is no need to keep track of the state in a
separate table.
Experimental and subject to future changes.
Add a way to redraw certain elements that are not redrawn while Nvim is waiting
for input, or currently have no API to do so. This API covers all that can be
done with the :redraw* commands, in addition to the following new features:
- Immediately move the cursor to a (non-current) window.
- Target a specific window or buffer to mark for redraw.
- Mark a buffer range for redraw (replaces nvim__buf_redraw_range()).
- Redraw the 'statuscolumn'.
- Added `@inlinedoc` so single use Lua types can be inlined into the
functions docs. E.g.
```lua
--- @class myopts
--- @inlinedoc
---
--- Documentation for some field
--- @field somefield integer
--- @param opts myOpts
function foo(opts)
end
```
Will be rendered as
```
foo(opts)
Parameters:
- {opts} (table) Object with the fields:
- somefield (integer) Documentation
for some field
```
- Marked many classes with with `@nodoc` or `(private)`.
We can eventually introduce these when we want to.
Problem: luals returns stricter diagnostics with bundled luarc.json
Solution: Improve some function and type annotations:
* use recognized uv.* types
* disable diagnostic for global `vim` in shared.lua
* docs: don't start comment lines with taglink (otherwise LuaLS will interpret it as a type)
* add type alias for lpeg pattern
* fix return annotation for `vim.secure.trust`
* rename local Range object in vim.version (shadows `Range` in vim.treesitter)
* fix some "missing fields" warnings
* add missing required fields for test functions in eval.lua
* rename lsp meta files for consistency
* docs(lua): teach lua2dox how to table
* docs(lua): teach gen_vimdoc.py about local functions
No more need to mark local functions with @private
* docs(lua): mention @nodoc and @meta in dev-lua-doc
* fixup!
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
---------
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
perf(lsp): load buffer contents once when processing semantic token responses
Using _get_line_byte_from_position() for each token's boundaries was a
pretty huge bottleneck, since that function would load individual buffer
lines via nvim_buf_get_lines() (plus a lot of extra overhead). So each
token caused two calls to nvim_buf_get_lines() (once for the start
position, and once for the end position).
For semantic tokens, we only attach to buffers that have already been
loaded, so we can safely just get all the lines for the entire buffer at
once, and lift the rest of the _get_line_byte_from_position()
implementation directly while bypassing the part that loads the buffer
line.
While I was looking at get_lines (used by _get_line_byte_from_position),
I noticed that we were checking for non-file URIs before we even looked
to see if we already had the buffer loaded. Moving the buffer-loaded
check to be the first thing done in get_lines() more than halved the
average time spent transforming the token list into highlight ranges vs
when it was still using _get_line_byte_from_position. I ended up
improving that loop more by not using get_lines, but figured the
performance improvement it provided was worth leaving in.
This commit replaces the usage of math.floor((lo + hi) / 2) with the faster and equivalent bit.rshift(lo + hi, 1) for calculating the midpoint in binary search.
feat(lsp)!: change semantic token highlighting
Change the default highlights used, and add more highlights per token.
Add an LspTokenUpdate event and a highlight_token function.
:Inspect now shows any highlights applied by token highlighting rules,
default or user-defined.
BREAKING CHANGE: change the default highlight groups used by semantic
token highlighting.
redraw! redraws the entire screen instead of just the windows with
the buffer which were actually changed.
I considered trying to calculating the range for the delta
but it looks tricky. Could a follow-up.
Fixes#21543
This should provide a better user experience when appending or prepending text to a word that has a semantic token extmark. More often than not, the appended/prepended text to the word will end up becoming part of the token anyway, so just use that extmark as the user types.
Apply semantic token modifiers as separate extmarks with corresponding
highlight groups (e.g., `@readonly`). This is a low-effort PR to enable
the most common use cases (applying, e.g., italics or backgrounds on top
of type highlights; language-specific fallbacks like `@global.lua` are
also available). This can be replaced by more complicated selector-style
themes later on.
Instead of testing for every possible modifier type, only test bits up
to the highest set in the token array. Saves many bit ops and
comparisons when there are no modifiers or when the highest set bit is a
lower bit than the highest possible in the legend on average.
Can be further simplified when non-luaJIT gets the full bit module (see #21222)
