Subject: Help Needed – MeshAgent 32-bit Build Crashes with Heap Corruption #295
Description
Hi MeshCentral team and fellow developers,
I'm currently working with the open-source MeshAgent project. I use Visual Studio 2022 with MSVC v143 (latest version), and my operating system is Windows 10 Enterprise (22H2).
When I build MeshAgent as a 64-bit application, everything runs fine as a Windows service. However, when I build it as a 32-bit application, the service crashes with the following exception:
Unhandled exception at 0x77716D23 (ntdll.dll) in User.exe.32472.dmp: 0xC0000374: A heap has been corrupted (parameters: 0x77753960)
I’ve opened the .dmp file in Visual Studio and traced the crash. The issue seems to happen inside the MeshAgent_Start() function, specifically when calling ILibStartChain(agentHost->chain).
From the call stack, the crash occurs during the cleanup phase when DestroyHandler() is called on a module, followed by ILibMemory_Free() and _free_dbg() in the CRT. Eventually, it fails at HeapFree() with a corrupted heap.
Here’s a quick summary of the flow:
ServiceMain() calls MeshAgent_Start()
Inside MeshAgent_Start(), ILibStartChain() is called
During cleanup, DestroyHandler() is triggered
Then ILibMemory_Free() → _free_dbg() → _free_base() → HeapFree() → crash
Details:
In the ServiceMain.c file at void WINAPI ServiceMain(DWORD argc, LPTSTR *argv):
...
__try
{
agent = MeshAgent_Create(0);
agent->serviceReserved = 1;
MeshAgent_Start(agent, g_serviceArgc, g_serviceArgv); => The error in the code originates from the call stack.
agent = NULL;
}
__except (ILib_WindowsExceptionFilterEx(GetExceptionCode(), GetExceptionInformation(), &winException))
{
ILib_WindowsExceptionDebugEx(&winException);
}
...
In the agentcore.c file at int MeshAgent_Start(MeshAgentHostContainer *agentHost, int paramLen, char **param):
...
// Check to see if we are running as just a JavaScript Engine
if (agentHost->meshCoreCtx_embeddedScript != NULL || (paramLen >= 2 && ILibString_EndsWith(param[1], -1, ".js", 3) != 0) || (paramLen >= 2 && ILibString_EndsWith(param[1], -1, ".zip", 4) != 0))
{
// We are acting as a scripting engine
ILibChain_RunOnMicrostackThreadEx(agentHost->chain, MeshAgent_ScriptMode_Dispatched, reserved);
ILibStartChain(agentHost->chain);
agentHost->chain = NULL;
}
else
{
// We are acting as an Agent
ILibChain_RunOnMicrostackThreadEx(agentHost->chain, MeshAgent_AgentMode_Dispatched, reserved);
ILibStartChain(agentHost->chain); => The error in the code originates from the call stack.
agentHost->chain = NULL;
...
In the ILibParsers.c file at ILibExportMethod void ILibStartChain(void Chain):
...
chain->node = ILibLinkedList_GetNode_Head(((ILibBaseChain)Chain)->Links);
if(chain->node != NULL) {chain->node = ILibLinkedList_GetNextNode(chain->node);} // Skip the base timer which is the first element in the chain.
//
// Set the Terminate Flag to 1, so that ILibIsChainBeingDestroyed returns non-zero when we start cleanup
//
((struct ILibBaseChain*)Chain)->TerminateFlag = 1;
while(chain->node!=NULL && (module=(ILibChain_Link*)ILibLinkedList_GetDataFromNode(chain->node))!=NULL)
{
//
// If this module has a destroy method, call it.
//
if (module->DestroyHandler != NULL) module->DestroyHandler((void*)module); => The error in the code originates from the call stack.
ILibMemory_Free(module->MetaData);
//
// After calling the Destroy, we free the module and move to the next
//
ILibChain_FreeLink(module);
chain->node = ILibLinkedList_Remove(chain->node);
}
...
In the debug_heap.cpp file at extern "C" __declspec(noinline) void __cdecl _free_dbg(void* const block, int const block_use):
extern "C" __declspec(noinline) void __cdecl _free_dbg(void* const block, int const block_use)
{
__acrt_lock(__acrt_heap_lock);
__try
{
// If a block use was provided, use it; if the block use was not known,
// use the block use stored in the header. (For example, the block use
// is not known when this function is called by operator delete because
// the heap lock must be acquired to access the block header.)
int const actual_use{block_use == _UNKNOWN_BLOCK && block != nullptr
? header_from_block(block)->_block_use
: block_use};
free_dbg_nolock(block, actual_use); => The error in the code originates from the call stack.
}
__finally
{
__acrt_unlock(__acrt_heap_lock);
}
}
In the debug_heap.cpp file at static void __cdecl free_dbg_nolock(void* const block,int const block_use) throw():
// Optionally reclaim memory:
if ((_crtDbgFlag & _CRTDBG_DELAY_FREE_MEM_DF) == 0)
{
// Unlink this allocation from the global linked list:
if (header->_block_header_next)
{
header->_block_header_next->_block_header_prev = header->_block_header_prev;
}
else
{
_ASSERTE(__acrt_last_block == header);
__acrt_last_block = header->_block_header_prev;
}
if (header->_block_header_prev)
{
header->_block_header_prev->_block_header_next = header->_block_header_next;
}
else
{
_ASSERTE(__acrt_first_block == header);
__acrt_first_block = header->_block_header_next;
}
memset(header, dead_land_fill, sizeof(_CrtMemBlockHeader) + header->_data_size + no_mans_land_size);
_free_base(header); => The error in the code originates from the call stack.
}
else
{
header->_block_use = _FREE_BLOCK;
// Keep memory around as dead space:
memset(block_from_header(header), dead_land_fill, header->_data_size);
}
In the free_base.cpp file at extern "C" void __declspec(noinline) __cdecl _free_base(void* const block):
// This function implements the logic of free(). It is called directly by the
// free() function in the Release CRT, and it is called by the debug heap in the
// Debug CRT.
//
// This function must be marked noinline, otherwise free and
// _free_base will have identical COMDATs, and the linker will fold
// them when calling one from the CRT. This is necessary because free
// needs to support users patching in custom implementations.
extern "C" void __declspec(noinline) __cdecl _free_base(void* const block)
{
if (block == nullptr)
{
return;
}
if (!HeapFree(select_heap(block), 0, block)) => The error in the code originates from the call stack.
{
errno = __acrt_errno_from_os_error(GetLastError());
}
}
This only happens in the 32-bit build. The 64-bit build works perfectly.
Could this be a memory alignment issue, a pointer mismatch, or something specific to the 32-bit heap behavior? I would really appreciate any insights or suggestions on how to fix or debug this issue.
Thanks so much in advance!
Best regards, Viet