LuaLS processes the wrong LuaDoc for Penlight tablex.filter

[FunctionDJ]

How are you using the lua-language-server?

Visual Studio Code Extension (sumneko.lua)

Which OS are you using?

Windows

What is the issue affecting?

Libraries

Expected Behaviour

local tablex = require("pl.tablex")

tablex.filter(
    {},
    function() end -- no linting error
)

Actual Behaviour

local tablex = require("pl.tablex")

tablex.filter(
    {},
    function() end -- Cannot assign `function` to parameter `"()"|"<"|"<="|"=="|">"...(+5)`. (...)
)

Reproduction steps

1. Setup LuaLS and copy example code into a .lua file
2. Install Penlight using the Addon Manager

Additional Notes

Here's an extract of the LuaDoc. LuaLS seems to always pick the second definition instead of the first one which has the annotations for using a generic function.

---@generic T, A
---@param t T[]
---@param pred fun(val: T, arg: A): any
---@param arg A
---@return T[]
---@nodiscard
function tablex.filter(t, pred, arg) end

---filter an array's values using a predicate function
---@generic T
---@param t T[] -- a list-like table
---@param pred pl.BoolBinOpString -- a boolean function
---@param arg any -- optional argument to be passed as second argument of the predicate
---@return T[]
---@nodiscard
function tablex.filter(t, pred, arg) end

https://github.com/goldenstein64/pl-definitions/blob/ef73834d7b54b989a4decf9f4598b83b3c9ef621/library/pl/tablex.lua#L545

Log File

I set --loglevel=trace and added the code to an empty file while having tail running on the log but there are no relevant details.

$ tail -f ./file_home_function_git_dragdown-modules.log
[13:59:35.741][debug][#0:script/vm/node.lua:490]: clearNodeCache
[13:59:35.741][trace][#0:script/files.lua:286]: Set text:       untitled:Untitled-1     takes   0.00012299999999987     sec.
[13:59:35.741][trace][#0:script/files.lua:695]: Compile State:  untitled:Untitled-1
[13:59:35.843][debug][#0:script/provider/provider.lua:1120]: textDocument/semanticTokens/range
[13:59:35.857][debug][#0:script/provider/diagnostic.lua:330]: publishDiagnostics        untitled:Untitled-1     2
[13:59:40.272][debug][#0:script/vm/node.lua:490]: clearNodeCache
[13:59:40.273][trace][#0:script/files.lua:286]: Set text:       untitled:Untitled-1     takes   0.00021399999999971     sec.
[13:59:40.373][trace][#0:script/files.lua:695]: Compile State:  untitled:Untitled-1
[13:59:40.374][debug][#0:script/provider/provider.lua:1120]: textDocument/semanticTokens/range
[13:59:40.381][info] [#0:script/provider/diagnostic.lua:171]: clearDiagnostics  untitled:Untitled-1
[13:59:47.118][debug][#0:script/vm/node.lua:490]: clearNodeCache
[13:59:47.118][trace][#0:script/files.lua:286]: Set text:       untitled:Untitled-1     takes   0.00020199999999981     sec.
[13:59:47.219][trace][#0:script/files.lua:695]: Compile State:  untitled:Untitled-1
[13:59:47.220][debug][#0:script/provider/provider.lua:1120]: textDocument/semanticTokens/range
[13:59:47.232][debug][#0:script/provider/diagnostic.lua:330]: publishDiagnostics        untitled:Untitled-1     2

[tomlau10]

explanation of this behavior (?)

AFAIK, the server will try to type narrow the function signature when applicable.
One of the criteria is number of min/max params

lua-language-server/script/vm/function.lua

Lines 25 to 65 in 3a65772

	function vm.countParamsOfFunction(func)
	local min = 0
	local max = 0
	local def = 0
	if func.type == 'function' then
	if func.args then
	max = #func.args
	def = max
	for i = #func.args, 1, -1 do
	local arg = func.args[i]
	if arg.type == '...' then
	max = math.huge
	elseif arg.type == 'self'
	and i == 1 then
	min = i
	break
	elseif getDocParam(arg)
	and not vm.compileNode(arg):isNullable() then
	min = i
	break
	end
	end
	end
	end
	if func.type == 'doc.type.function' then
	if func.args then
	max = #func.args
	def = max
	for i = #func.args, 1, -1 do
	local arg = func.args[i]
	if arg.name and arg.name[1] =='...' then
	max = math.huge
	elseif not vm.compileNode(arg):isNullable() then
	min = i
	break
	end
	end
	end
	end
	return min, max, def
	end

• I added a debug print(min, max, def) just before the return line
  • for the 1st one it prints 3, 3, 3
  • for the 2nd one it prints 2, 3, 3
• note the difference of min param of the functions 😕
  => the server thinks that the 1st function signature requires 3 params, all are required and none are optional
  => since your code snippet only gives 2 params

  tablex.filter(
      {},
      function() end
  )

=> the server "filtered" it out, and narrowed to the 2nd one...

possible workaround

Looking at the code inside vm.countParamsOfFunction, it basically uses :isNullable() to determine the min value.

lua-language-server/script/vm/node.lua

Lines 162 to 178 in 3a65772

	function mt:isNullable()
	if self.optional then
	return true
	end
	if #self == 0 then
	return true
	end
	for _, c in ipairs(self) do
	if c.type == 'nil'
	or (c.type == 'global' and c.cate == 'type' and c.name == 'nil')
	or (c.type == 'global' and c.cate == 'type' and c.name == 'any')
	or (c.type == 'global' and c.cate == 'type' and c.name == '...') then
	return true
	end
	end
	return false
	end

• currently it only treats nil|any|... as optional, but not for generic param
  (I am not sure if this is intended behavior)
• however it also treats a manually ? marked param as optional
• so the workaround is to mark arg as optional with ? 🤔

---@param arg? A # note the `?` after `arg`

or if you prefer using |nil

---@param arg A|nil # note the `|nil` after `A`

=> then server can type narrow the function correctly

---

In short, the workaround is mark arg as optional.
Meanwhile whether generic param not counted as optional is a "feature" or bug, has to be clarified by maintainers. 😕

[FunctionDJ]

@tomlau10 Thanks a ton for this breakdown! Very insightful and comprehensible.

From the user-side, just passing any value as a third parameter (that can go unused in the predicate function) does indeed make LuaLS pick the correct type definition and doesn't yield a type warning.

local tablex = require("pl.tablex")

local doubled = tablex.filter(
    { 1, 2, 3 },
    function(num, unused) return num * 2 end, -- no linting error and you'd usually not even specify the unused param
    nil
)