Problem: No way to check the version of a treesitter parser.
Solution: Add version metadata (ABI 15 parsers only) as well as parser state count and supertype information (ABI 15) in `vim.treesitter.language.inspect()`. Also graduate the `abi_version` field, as this is now the official upstream name.
---------
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
Problem: It's difficult to navigate large structured text files (vim
help, checkhealth, Markdown).
Solution: Support `gO` for table of contents and `]]`/`[[` for moving
between headings for all these filetypes using treesitter queries.
Refactor: colorization of highlight groups is moved to the `help` ftplugin
while headings-related functionality is implemented in a private
`vim.treesitter` module for possible future use for other filetypes.
Problem:
`vim.validate()` takes two forms when it only needs one.
Solution:
- Teach the fast form all the features of the spec form.
- Deprecate the spec form.
- General optimizations for both forms.
- Add a `message` argument which can be used alongside or in place
of the `optional` argument.
**Problems:**
- `vim.treesitter.language.inspect()` returns duplicate
symbol names, sometimes up to 6 of one kind in the case of `markdown`
- The list-like `symbols` table can have holes and is thus not even a
valid msgpack table anyway, mentioned in a test
**Solution:** Return symbols as a map, rather than a list, where field
names are the names of the symbol. The boolean value associated with the
field encodes whether or not the symbol is named.
Note that anonymous nodes are surrounded with double quotes (`"`) to
prevent potential collisions with named counterparts that have the same
identifier.
Problem: No clear way to check whether parsers are available for a given
language.
Solution: Make `language.add()` return `true` if a parser was
successfully added and `nil` otherwise. Use explicit `assert` instead of
relying on thrown errors.
Problem: Language names are only registered for filetype<->language
lookups when parsers are actually loaded; this means users cannot rely
on `vim.treesitter.language.get_lang()` or `get_filetypes()` to return
the correct value when language and filetype coincide and always need to
add explicit fallbacks.
Solution: Always return the language name as valid filetype in
`get_filetypes()`, and default to the filetype in `get_lang()`. Document
this behavior.
Problem: Installing treesitter parser is hard (harder than
climbing to heaven).
Solution: Add optional support for wasm parsers with `wasmtime`.
Notes:
* Needs to be enabled by setting `ENABLE_WASMTIME` for tree-sitter and
Neovim. Build with
`make CMAKE_EXTRA_FLAGS=-DENABLE_WASMTIME=ON
DEPS_CMAKE_FLAGS=-DENABLE_WASMTIME=ON`
* Adds optional Rust (obviously) and C11 dependencies.
* Wasmtime comes with a lot of features that can negatively affect
Neovim performance due to library and symbol table size. Make sure to
build with minimal features and full LTO.
* To reduce re-compilation times, install `sccache` and build with
`RUSTC_WRAPPER=<path/to/sccache> make ...`
* fix(treesitter): enforce lowercase language names
Problem: On case-insensitive file systems (e.g., macOS), `has_parser`
will return `true` for uppercase aliases, which will then try to inject
the uppercase language unsuccessfully.
Solution: Enforce and assume parser names to be lowercase when
resolving language names.
- Added `@inlinedoc` so single use Lua types can be inlined into the
functions docs. E.g.
```lua
--- @class myopts
--- @inlinedoc
---
--- Documentation for some field
--- @field somefield integer
--- @param opts myOpts
function foo(opts)
end
```
Will be rendered as
```
foo(opts)
Parameters:
- {opts} (table) Object with the fields:
- somefield (integer) Documentation
for some field
```
- Marked many classes with with `@nodoc` or `(private)`.
We can eventually introduce these when we want to.
The benefit of this is that users only pay for what they use. If e.g.
only `vim.lsp.buf_get_clients()` is called then they don't need to load
all modules under `vim.lsp` which could lead to significant startuptime
saving.
Also `vim.lsp.module` is a bit nicer to user compared to
`require("vim.lsp.module")`.
This isn't used for some nested modules such as `filetype` as it breaks
tests with error messages such as "attempt to index field 'detect'".
It's not entirely certain the reason for this, but it is likely it is
due to filetype being precompiled which would imply deferred loading
isn't needed for performance reasons.
Improve error messages for `:InspectTree`, when no parsers are available
for the current buffer and filetype. We can show more informative and
helpful error message for users (e.g., which lang was searched for):
```
... No parser available for the given buffer:
+... no parser for 'custom_ft' language, see :help treesitter-parsers
```
Also improve the relevant docs for *treesitter-parsers*.
* docs(lua): teach lua2dox how to table
* docs(lua): teach gen_vimdoc.py about local functions
No more need to mark local functions with @private
* docs(lua): mention @nodoc and @meta in dev-lua-doc
* fixup!
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
---------
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Problem:
Codebase inconsistently binds vim.api onto a or api.
Solution:
Use api everywhere. a as an identifier is too short to have at the
module level.
Problem:
vim.treesitter does not know how to map a specific filetype to a parser.
This creates problems since in a few places (including in vim.treesitter itself), the filetype is incorrectly used in place of lang.
Solution:
Add an API to enable this:
- Add vim.treesitter.language.add() as a replacement for vim.treesitter.language.require_language().
- Optional arguments are now passed via an opts table.
- Also takes a filetype (or list of filetypes) so we can keep track of what filetypes are associated with which langs.
- Deprecated vim.treesitter.language.require_language().
- Add vim.treesitter.language.get_lang() which returns the associated lang for a given filetype.
- Add vim.treesitter.language.register() to associate filetypes to a lang without loading the parser.
Problem: Some injections (like markdown) allow specifying arbitrary
language names for code blocks, which may be lead to errors when
looking for a corresponding parser in runtime path.
Solution: Validate that the language name only contains alphanumeric
characters and `_` (e.g., for `c_sharp`) and error otherwise.
When trying to load a language parser, escape the value of
the language.
With language injection, the language might be picked up from the
buffer. If this value is erroneous it can cause `nvim_get_runtime_file`
to hard error.
E.g., the markdown expression `~~~{` will extract '{' as a language and
then try to get the parser using `parser/{*` as the pattern.
The official developer documentation in in :h dev-lua-doc specifies to
use "--@" for special/magic tokens. However, this format is not
consistent with EmmyLua notation (used by some Lua language servers) nor
with the C version of the magic docstring tokens which use three comment
characters.
Further, the code base is currently split between usage of "--@",
"---@", and "--- @". In an effort to remain consistent, change all Lua
magic tokens to use "---@" and update the developer documentation
accordingly.
