Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Invalid allFunctionsWithMain yml files #2069

Open
eaubin opened this issue Feb 6, 2025 · 4 comments
Open

Invalid allFunctionsWithMain yml files #2069

eaubin opened this issue Feb 6, 2025 · 4 comments

Comments

@eaubin
Copy link

eaubin commented Feb 6, 2025 •
edited
Loading

Sometimes I'll see allFunctionsWithMain yaml files with errors. For example, loading one of the sqlite files with python yaml.safe_load() errors with:

ERROR:__main__:Error reading ../oss-fuzz/build/out/sqlite3/inspector/allFunctionsWithMain-19-fa37JncCHr.yaml
ERROR:root:while parsing a block mapping
  in "../oss-fuzz/build/out/sqlite3/inspector/allFunctionsWithMain-19-fa37JncCHr.yaml", line 447239, column 7
expected <block end>, but found '<scalar>'
  in "../oss-fuzz/build/out/sqlite3/inspector/allFunctionsWithMain-19-fa37JncCHr.yaml", line 447902, column 8

The text around those lines look like:

447230	              BranchSideFuncs:
447231	                - sqlite3ResolveExprListNames
447232	                - __assert_fail
447233	                - __assert_fail
447234	                - sq---
447235	Fuzzer filename: ''
447236	All functions:
447237	  Function list name: All functions
447238	  Elements:
447239	    - functionName:    sqlite3TreeViewTrigger
447240	      functionSourceFile: '/src/sqlite3/bld/sqlite3.c'
447241	      linkageType:     InternalLinkage
447242	      functionLinenumber: 34115
447243	      functionLinenumberEnd: 34131

and

447896	        - Branch String:   'sqlite3.c:109097,9'
447897	          Branch Sides:
447898	            - BranchSide:      'sqlite3.c:109098,12'
447899	              BranchSideFuncs:
447900	                - sqlite3ResolveExprNames
447901	                - sqlite3ResolveExprNames
447902	       iewPop
447903	                - sqlite3TreeViewPush
447904	                - sqlite3TreeViewLine
447905	                - sqlite3TreeViewPush
447906	                - sqlite3TreeViewTriggerStep
447907	                - sqlite3TreeViewPop

more of the text is in this gist, but the whole thing is too big to upload.
@DavidKorczynski
Copy link
Contributor

Hmm, do you have a reproducer?

@eaubin
Copy link
Author

eaubin commented Feb 7, 2025

I don't have a development workflow, this was from oss-fuzz c3506f867184ea1aff7b37753de973f6fd46c47f run Jan 3. I'm running the introspector through oss-fuzz and it's hard to keep track of what versions of containers actually ran. Are the files available on https://storage.googleapis.com/oss-fuzz-introspector? With the random filename I can't check the public runs. I'll have to try running it again.

@eaubin
Copy link
Author

eaubin commented Feb 7, 2025 •
edited
Loading

I get similar problems a fresh pull/run:

$ git pull
remote: Enumerating objects: 339, done.
remote: Counting objects: 100% (174/174), done.
remote: Compressing objects: 100% (84/84), done.
remote: Total 339 (delta 146), reused 90 (delta 90), pack-reused 165 (from 5)
Receiving objects: 100% (339/339), 220.50 KiB | 4.41 MiB/s, done.
Resolving deltas: 100% (163/163), completed with 19 local objects.
From https://github.com/google/oss-fuzz
   5a9322260..d5c068896  master     -> origin/master
 * [new branch]          hla        -> origin/hla
Updating 5a9322260..d5c068896

$ python3 infra/helper.py introspector sqlite3 --seconds=30
# too big for issues, see https://gist.github.com/eaubin/d35c8b38630c7000322b2b805964e1b1

$ python3 -c 'import yaml; f=open("oss-fuzz/build/out/sqlite3/inspector/allFunctionsWithMain-19-fa37JncCHr.yaml"); yaml.safe_load(f)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 162, in safe_load
    return load(stream, SafeLoader)
  File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/usr/lib/python3/dist-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 111, in compose_sequence_node
    node.value.append(self.compose_node(node, index))
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 111, in compose_sequence_node
    node.value.append(self.compose_node(node, index))
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/lib/python3/dist-packages/yaml/composer.py", line 127, in compose_mapping_node
    while not self.check_event(MappingEndEvent):
  File "/usr/lib/python3/dist-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/usr/lib/python3/dist-packages/yaml/parser.py", line 428, in parse_block_mapping_key
    if self.check_token(KeyToken):
  File "/usr/lib/python3/dist-packages/yaml/scanner.py", line 115, in check_token
    while self.need_more_tokens():
  File "/usr/lib/python3/dist-packages/yaml/scanner.py", line 152, in need_more_tokens
    self.stale_possible_simple_keys()
  File "/usr/lib/python3/dist-packages/yaml/scanner.py", line 291, in stale_possible_simple_keys
    raise ScannerError("while scanning a simple key", key.mark,
yaml.scanner.ScannerError: while scanning a simple key
  in "oss-fuzz/build/out/sqlite3/inspector/allFunctionsWithMain-19-fa37JncCHr.yaml", line 1389842, column 3
could not find expected ':'
  in "oss-fuzz/build/out/sqlite3/inspector/allFunctionsWithMain-19-fa37JncCHr.yaml", line 1389962, column 25

The yaml looks like:

1389838               BranchSideFuncs: []
1389839       Callsites:
1389840         - Src:             'src/tclsqlite.c:832,3'
1389841           Dst:             __assert_fail
1389842   leaseMalloc
1389843                 - __assert_fail
1389844                 - __assert_fail
...
1389960                 - out2Prerelease
1389961                 - __assert_fail
1389962           - functionName:    DbUpdateHandler
1389963       functionSourceFile: '/src/sqlite3/src/tclsqlite.c'
1389964       linkageType:     InternalLinkage
1389965       functionLinenumber: 937
1389966       functionLinenumberEnd: 957
1389967       functionDepth:   0

@eaubin
Copy link
Author

eaubin commented Feb 8, 2025

There are similar problems in the larger files produced for curl and openssl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@eaubin @DavidKorczynski