Problem:
LSP spec uses the term "position encoding" where we say "offset encoding".
Solution:
- Rename it everywhere except `vim.lsp.Client.offset_encoding` (which would be breaking).
- Mention "position encoding" in the documentation for `vim.lsp.Client.offset_encoding`.
* deprecate old signatures
* move to new str_byteindex/str_utfindex signature
* use single-underscore name (double-underscore is reserved for Lua itself)
Problem:
Some language servers (e.g., rust-analyzer, texlab) are desynced when
the user deletes the entire contents of the buffer. This is due to the
discrepancy between how nvim computes diff and how nvim treats empty
buffer.
* diff: If the buffer became empty, then the diff includes the last
line's eol.
* empty buffer: Even if the buffer is empty, nvim regards it as having
a single empty line with eol.
Solution:
Add special case for diff computation when the buffer becomes empty so
that it does not include the eol of the last line.
Problem:
The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.
Solution:
Re-implement the flow using Lua, LPEG and treesitter.
- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
- LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
- LPEG for C parsing (see `scripts/cdoc_parser.lua`)
- Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
- Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
- `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
* docs(lua): teach lua2dox how to table
* docs(lua): teach gen_vimdoc.py about local functions
No more need to mark local functions with @private
* docs(lua): mention @nodoc and @meta in dev-lua-doc
* fixup!
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
---------
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Closes#16624
Fixes two issues with aligning the start position and end position to
codepoints when calculating the start and end range.
When aligning the start position:
* use aligned byte index to calculate character index rather than
the unadjusted byte
When aligning the end position:
* do not adjust the end byte if it falls on a UTF-8 codepoint
* align byte to the first byte of the next codepoint rather than the
last byte of the current codepoint
* compute character character end range on the aligned byte index
This commit also adds additional test coverage, including multibyte operations
that previously failed before this commit.
* use codeunits/points instead of byte ranges when applicable
* take into account different file formats when computing range and
sending text (dos, unix, and mac supported)
* add tests of incremental sync
