Function highlight_rust_tree_sitter_hybrid

pub fn highlight_rust_tree_sitter_hybrid(text: &str) -> Option<Vec<SyntaxToken>>

Combines Tree-sitter Rust tokens with lexical gap filling.

Returns None if parser setup/parsing fails or non-whitespace input yields no structural token. Otherwise ranges are clamped to UTF-8 boundaries, sorted, deduplicated, and made non-overlapping by semantic priority before and after lexical gap filling. Whitespace-only input returns Some(Vec::new()).

Examples

use ailloli_ui_editor::code::{highlight_rust_tree_sitter_hybrid, SyntaxKind};
let text = "fn main() { let n = 1; }";
let tokens = highlight_rust_tree_sitter_hybrid(text).unwrap();
assert!(tokens.iter().any(|t| t.kind == SyntaxKind::Function && &text[t.range.clone()] == "main"));
assert_eq!(highlight_rust_tree_sitter_hybrid("   "), Some(Vec::new()));