Skip to content

Commit c80c9b4

Browse files
authored
added logs
1 parent 17d1938 commit c80c9b4

28 files changed

+97829
-0
lines changed

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-03_12-40-34.log

+1,016
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-03_13-37-59.log

+1,207
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-03_13-53-48.log

+963
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-04_07-57-39.log

+3,744
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-26_13-25-51.log

+1,691
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-26_13-59-53.log

+207
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_13-26-37.log

+1,614
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_13-45-55.log

+1,614
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_16-18-59.log

+2,254
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_16-30-25.log

+287
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_16-31-21.log

+287
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_16-49-07.log

+287
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_17-06-21.log

+207
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_17-08-57.log

+207
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_17-27-19.log

+229
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_17-29-22.log

+229
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_17-45-01.log

+1,816
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_17-52-19.log

+31
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_18-18-08.log

+805
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_18-30-30.log

+349
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_18-35-08.log

+1,094
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_18-46-43.log

+1,151
Large diffs are not rendered by default.

2023-DL4KG_Automated-Tests-with-LLM-KGE-Bench/log/llm-kg-bench-run_2023-07-27_18-57-25.log

+20
Large diffs are not rendered by default.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
2023-07-27 18:59:20,783 [DEBUG] config used: {'tests': ['TurtleFriendCount'], 'models': [{'label': 'GPT-4', 'class': 'ModelGpt', 'params': [{'model': 'gpt-4'}]}], 'sizes': [1000, 2000], 'iterations': 1}
2+
2023-07-27 18:59:20,784 [INFO] test configuration: tests:['TurtleFriendCount']; iterations:1; sizes:[1000, 2000]; models:[{'label': 'GPT-4', 'class': 'ModelGpt', 'params': {'model': 'gpt-4'}}]
3+
2023-07-27 18:59:20,784 [INFO] Start for task 'TurtleFriendCount' version '2.0' at 2023-07-27 18:59:20.784710
4+
2023-07-27 18:59:20,784 [DEBUG] init model: class=ModelGpt; params={'model': 'gpt-4'}.
5+
2023-07-27 18:59:20,785 [INFO] model=GPT-gpt-4 ...
6+
2023-07-27 18:59:20,785 [INFO] size=1000
7+
2023-07-27 18:59:20,785 [INFO] iteration=1/1 ...
8+
2023-07-27 18:59:20,786 [INFO] evaluating model=GPT-4--GPT-gpt-4; time=2023-07-27 18:59:20.785988; size=1000; i=1
9+
2023-07-27 18:59:20,789 [DEBUG] sending to model "gpt-4" message(~997 chars): "[{'role': 'user', 'content': 'Please name the person which is known by the most persons according to the following RDF graph in turtle syntax. Give just the IRI of this person with most incomming links as answer, without abbreviation or explanation.@prefix ex: <http://example.org/people/> .\\n@prefix foaf: <http://xmlns.com/foaf/0.1/> .\\n\\nex:person-00000001 a foaf:Person ;\\n foaf:knows ex:person-00000002,\\n ex:person-00000003,\\n ex:person-00000004 .\\n\\nex:person-00000002 a foaf:Person ;\\n foaf:knows ex:person-00000003,\\n ex:person-00000004 .\\n\\nex:person-00000003 a foaf:Person ;\\n foaf:knows ex:person-00000004,\\n ex:person-00000005 .\\n\\nex:person-00000005 a foaf:Person ;\\n foaf:knows ex:person-00000001,\\n ex:person-00000006 .\\n\\nex:person-00000006 a foaf:Person ;\\n foaf:knows ex:person-00000001,\\n ex:person-00000002 .\\n\\nex:person-00000004 a foaf:Person ;\\n foaf:knows ex:person-00000005,\\n ex:person-00000006 .\\n\\n'}]"
10+
2023-07-27 18:59:20,792 [DEBUG] message='Request to OpenAI API' method=post path=https://api.openai.com/v1/chat/completions
11+
2023-07-27 18:59:20,792 [DEBUG] api_version=None data='{"model": "gpt-4", "messages": [{"role": "user", "content": "Please name the person which is known by the most persons according to the following RDF graph in turtle syntax. Give just the IRI of this person with most incomming links as answer, without abbreviation or explanation.@prefix ex: <http://example.org/people/> .\\n@prefix foaf: <http://xmlns.com/foaf/0.1/> .\\n\\nex:person-00000001 a foaf:Person ;\\n foaf:knows ex:person-00000002,\\n ex:person-00000003,\\n ex:person-00000004 .\\n\\nex:person-00000002 a foaf:Person ;\\n foaf:knows ex:person-00000003,\\n ex:person-00000004 .\\n\\nex:person-00000003 a foaf:Person ;\\n foaf:knows ex:person-00000004,\\n ex:person-00000005 .\\n\\nex:person-00000005 a foaf:Person ;\\n foaf:knows ex:person-00000001,\\n ex:person-00000006 .\\n\\nex:person-00000006 a foaf:Person ;\\n foaf:knows ex:person-00000001,\\n ex:person-00000002 .\\n\\nex:person-00000004 a foaf:Person ;\\n foaf:knows ex:person-00000005,\\n ex:person-00000006 .\\n\\n"}]}' message='Post details'
12+
2023-07-27 18:59:20,792 [DEBUG] Converted retries value: 2 -> Retry(total=2, connect=None, read=None, redirect=None, status=None)
13+
2023-07-27 18:59:20,793 [DEBUG] Starting new HTTPS connection (1): api.openai.com:443
14+
2023-07-27 18:59:21,996 [DEBUG] https://api.openai.com:443 "POST /v1/chat/completions HTTP/1.1" 200 None
15+
2023-07-27 18:59:21,997 [DEBUG] message='OpenAI API response' path=https://api.openai.com/v1/chat/completions processing_ms=1044 request_id=70bde868ca1a897f6fd42397a80cd12f response_code=200
16+
2023-07-27 18:59:21,999 [DEBUG] got from model "gpt-4" raw answer: {"id": "chatcmpl-7h0Uj1FgTntCYlcNSIAXGOlPyUgNo", "object": "chat.completion", "created": 1690484361, "model": "gpt-4-0613", "choices": [{"index": 0, "message": {"role": "assistant", "content": "http://example.org/people/person-00000001"}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 305, "completion_tokens": 10, "total_tokens": 315}}
17+
2023-07-27 18:59:21,999 [DEBUG] got from model "gpt-4-0613" answer(~41 chars): "http://example.org/people/person-00000001"
18+
2023-07-27 18:59:22,000 [DEBUG] evaluation done with result = {"startTime": "2023-07-27T18:59:20.789591", "modelId": "GPT--gpt-4--gpt-4-0613", "taskInfo": {"taskId": "TurtleFriendCount", "version": "2.0", "gitCommit": "dd716f229c6d9c627544a513aa00d953882a1f45"}, "scores": {"recall": 0, "brevity": 1, "precision": 0, "f1": 0, "noSpecialFriends": 1.0}, "info": {"persons": 6, "triples": 19, "graphSizeChar": 717, "specialNode": "http://example.org/people/person-00000004", "answerPersonCount": 1, "specialFriendsInAnswer": 0, "personsInAnswer": ["1"]}, "duration": "0:00:01.209971", "interactions": [["prompt", "2023-07-27T18:59:20.789596", "Please name the person which is known by the most persons according to the following RDF graph in turtle syntax. Give just the IRI of this person with most incomming links as answer, without abbreviation or explanation.@prefix ex: <http://example.org/people/> .\n@prefix foaf: <http://xmlns.com/foaf/0.1/> .\n\nex:person-00000001 a foaf:Person ;\n foaf:knows ex:person-00000002,\n ex:person-00000003,\n ex:person-00000004 .\n\nex:person-00000002 a foaf:Person ;\n foaf:knows ex:person-00000003,\n ex:person-00000004 .\n\nex:person-00000003 a foaf:Person ;\n foaf:knows ex:person-00000004,\n ex:person-00000005 .\n\nex:person-00000005 a foaf:Person ;\n foaf:knows ex:person-00000001,\n ex:person-00000006 .\n\nex:person-00000006 a foaf:Person ;\n foaf:knows ex:person-00000001,\n ex:person-00000002 .\n\nex:person-00000004 a foaf:Person ;\n foaf:knows ex:person-00000005,\n ex:person-00000006 .\n\n"], ["response", "2023-07-27T18:59:21.999562", "http://example.org/people/person-00000001"]]}
19+
2023-07-27 18:59:22,000 [INFO] scores: {'recall': 0, 'brevity': 1, 'precision': 0, 'f1': 0, 'noSpecialFriends': 1.0}
20+
2023-07-27 18:59:22,006 [INFO] size=2000
21+
2023-07-27 18:59:22,006 [INFO] iteration=1/1 ...
22+
2023-07-27 18:59:22,007 [INFO] evaluating model=GPT-4--GPT-gpt-4; time=2023-07-27 18:59:22.007109; size=2000; i=1
23+
2023-07-27 18:59:22,013 [DEBUG] sending to model "gpt-4" message(~2095 chars): "[{'role': 'user', 'content': 'Please name the person which is known by the most persons according to the following RDF graph in turtle syntax. Give just the IRI of this person with most incomming links as answer, without abbreviation or explanation.@prefix ex: <http://example.org/people/> .\\n@prefix foaf: <http://xmlns.com/foaf/0.1/> .\\n\\nex:person-00000001 a foaf:Person ;\\n foaf:knows ex:person-00000002,\\n ex:person-00000003 .\\n\\nex:person-00000002 a foaf:Person ;\\n foaf:knows ex:person-00000003,\\n ex:person-00000004 .\\n\\nex:person-00000003 a foaf:Person ;\\n foaf:knows ex:person-00000004,\\n ex:person-00000005 .\\n\\nex:person-00000005 a foaf:Person ;\\n foaf:knows ex:person-00000006,\\n ex:person-00000007 .\\n\\nex:person-00000006 a foaf:Person ;\\n foaf:knows ex:person-00000007,\\n ex:person-00000008 .\\n\\nex:person-00000007 a foaf:Person ;\\n foaf:knows ex:person-00000004,\\n ex:person-00000008,\\n ex:person-00000009 .\\n\\nex:person-00000008 a foaf:Person ;\\n foaf:knows ex:person-00000004,\\n ex:person-00000009,\\n ex:person-0000000a .\\n\\nex:person-00000009 a foaf:Person ;\\n foaf:knows ex:person-00000004,\\n ex:person-0000000a,\\n ex:person-0000000b .\\n\\nex:person-0000000a a foaf:Person ;\\n foaf:knows ex:person-0000000b,\\n ex:person-0000000c .\\n\\nex:person-0000000b a foaf:Person ;\\n foaf:knows ex:person-0000000c,\\n ex:person-0000000d .\\n\\nex:person-0000000c a foaf:Person ;\\n foaf:knows ex:person-0000000d,\\n ex:person-0000000e .\\n\\nex:person-0000000d a foaf:Person ;\\n foaf:knows ex:person-0000000e,\\n ex:person-0000000f .\\n\\nex:person-0000000e a foaf:Person ;\\n foaf:knows ex:person-0000000f,\\n ex:person-00000010 .\\n\\nex:person-0000000f a foaf:Person ;\\n foaf:knows ex:person-00000001,\\n ex:person-00000010 .\\n\\nex:person-00000010 a foaf:Person ;\\n foaf:knows ex:person-00000001,\\n ex:person-00000002 .\\n\\nex:person-00000004 a foaf:Person ;\\n foaf:knows ex:person-00000005,\\n ex:person-00000006 .\\n\\n'}]"
24+
2023-07-27 18:59:22,014 [DEBUG] message='Request to OpenAI API' method=post path=https://api.openai.com/v1/chat/completions
25+
2023-07-27 18:59:22,014 [DEBUG] api_version=None data='{"model": "gpt-4", "messages": [{"role": "user", "content": "Please name the person which is known by the most persons according to the following RDF graph in turtle syntax. Give just the IRI of this person with most incomming links as answer, without abbreviation or explanation.@prefix ex: <http://example.org/people/> .\\n@prefix foaf: <http://xmlns.com/foaf/0.1/> .\\n\\nex:person-00000001 a foaf:Person ;\\n foaf:knows ex:person-00000002,\\n ex:person-00000003 .\\n\\nex:person-00000002 a foaf:Person ;\\n foaf:knows ex:person-00000003,\\n ex:person-00000004 .\\n\\nex:person-00000003 a foaf:Person ;\\n foaf:knows ex:person-00000004,\\n ex:person-00000005 .\\n\\nex:person-00000005 a foaf:Person ;\\n foaf:knows ex:person-00000006,\\n ex:person-00000007 .\\n\\nex:person-00000006 a foaf:Person ;\\n foaf:knows ex:person-00000007,\\n ex:person-00000008 .\\n\\nex:person-00000007 a foaf:Person ;\\n foaf:knows ex:person-00000004,\\n ex:person-00000008,\\n ex:person-00000009 .\\n\\nex:person-00000008 a foaf:Person ;\\n foaf:knows ex:person-00000004,\\n ex:person-00000009,\\n ex:person-0000000a .\\n\\nex:person-00000009 a foaf:Person ;\\n foaf:knows ex:person-00000004,\\n ex:person-0000000a,\\n ex:person-0000000b .\\n\\nex:person-0000000a a foaf:Person ;\\n foaf:knows ex:person-0000000b,\\n ex:person-0000000c .\\n\\nex:person-0000000b a foaf:Person ;\\n foaf:knows ex:person-0000000c,\\n ex:person-0000000d .\\n\\nex:person-0000000c a foaf:Person ;\\n foaf:knows ex:person-0000000d,\\n ex:person-0000000e .\\n\\nex:person-0000000d a foaf:Person ;\\n foaf:knows ex:person-0000000e,\\n ex:person-0000000f .\\n\\nex:person-0000000e a foaf:Person ;\\n foaf:knows ex:person-0000000f,\\n ex:person-00000010 .\\n\\nex:person-0000000f a foaf:Person ;\\n foaf:knows ex:person-00000001,\\n ex:person-00000010 .\\n\\nex:person-00000010 a foaf:Person ;\\n foaf:knows ex:person-00000001,\\n ex:person-00000002 .\\n\\nex:person-00000004 a foaf:Person ;\\n foaf:knows ex:person-00000005,\\n ex:person-00000006 .\\n\\n"}]}' message='Post details'
26+
2023-07-27 18:59:23,306 [DEBUG] https://api.openai.com:443 "POST /v1/chat/completions HTTP/1.1" 200 None
27+
2023-07-27 18:59:23,307 [DEBUG] message='OpenAI API response' path=https://api.openai.com/v1/chat/completions processing_ms=1129 request_id=96ff65b67cde7648368b5cafe4160b51 response_code=200
28+
2023-07-27 18:59:23,307 [DEBUG] got from model "gpt-4" raw answer: {"id": "chatcmpl-7h0UkfvcDUrfXDIBnYa0DEsAzYtuX", "object": "chat.completion", "created": 1690484362, "model": "gpt-4-0613", "choices": [{"index": 0, "message": {"role": "assistant", "content": "http://example.org/people/person-00000004"}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 701, "completion_tokens": 10, "total_tokens": 711}}
29+
2023-07-27 18:59:23,307 [DEBUG] got from model "gpt-4-0613" answer(~41 chars): "http://example.org/people/person-00000004"
30+
2023-07-27 18:59:23,308 [DEBUG] evaluation done with result = {"startTime": "2023-07-27T18:59:22.013554", "modelId": "GPT--gpt-4--gpt-4-0613", "taskInfo": {"taskId": "TurtleFriendCount", "version": "2.0", "gitCommit": "dd716f229c6d9c627544a513aa00d953882a1f45"}, "scores": {"recall": 1, "brevity": 1, "precision": 1.0, "f1": 1.0, "noSpecialFriends": 1.0}, "info": {"persons": 16, "triples": 51, "graphSizeChar": 1773, "specialNode": "http://example.org/people/person-00000004", "answerPersonCount": 1, "specialFriendsInAnswer": 0, "personsInAnswer": ["4"]}, "duration": "0:00:01.294270", "interactions": [["prompt", "2023-07-27T18:59:22.013560", "Please name the person which is known by the most persons according to the following RDF graph in turtle syntax. Give just the IRI of this person with most incomming links as answer, without abbreviation or explanation.@prefix ex: <http://example.org/people/> .\n@prefix foaf: <http://xmlns.com/foaf/0.1/> .\n\nex:person-00000001 a foaf:Person ;\n foaf:knows ex:person-00000002,\n ex:person-00000003 .\n\nex:person-00000002 a foaf:Person ;\n foaf:knows ex:person-00000003,\n ex:person-00000004 .\n\nex:person-00000003 a foaf:Person ;\n foaf:knows ex:person-00000004,\n ex:person-00000005 .\n\nex:person-00000005 a foaf:Person ;\n foaf:knows ex:person-00000006,\n ex:person-00000007 .\n\nex:person-00000006 a foaf:Person ;\n foaf:knows ex:person-00000007,\n ex:person-00000008 .\n\nex:person-00000007 a foaf:Person ;\n foaf:knows ex:person-00000004,\n ex:person-00000008,\n ex:person-00000009 .\n\nex:person-00000008 a foaf:Person ;\n foaf:knows ex:person-00000004,\n ex:person-00000009,\n ex:person-0000000a .\n\nex:person-00000009 a foaf:Person ;\n foaf:knows ex:person-00000004,\n ex:person-0000000a,\n ex:person-0000000b .\n\nex:person-0000000a a foaf:Person ;\n foaf:knows ex:person-0000000b,\n ex:person-0000000c .\n\nex:person-0000000b a foaf:Person ;\n foaf:knows ex:person-0000000c,\n ex:person-0000000d .\n\nex:person-0000000c a foaf:Person ;\n foaf:knows ex:person-0000000d,\n ex:person-0000000e .\n\nex:person-0000000d a foaf:Person ;\n foaf:knows ex:person-0000000e,\n ex:person-0000000f .\n\nex:person-0000000e a foaf:Person ;\n foaf:knows ex:person-0000000f,\n ex:person-00000010 .\n\nex:person-0000000f a foaf:Person ;\n foaf:knows ex:person-00000001,\n ex:person-00000010 .\n\nex:person-00000010 a foaf:Person ;\n foaf:knows ex:person-00000001,\n ex:person-00000002 .\n\nex:person-00000004 a foaf:Person ;\n foaf:knows ex:person-00000005,\n ex:person-00000006 .\n\n"], ["response", "2023-07-27T18:59:23.307824", "http://example.org/people/person-00000004"]]}
31+
2023-07-27 18:59:23,308 [INFO] scores: {'recall': 1, 'brevity': 1, 'precision': 1.0, 'f1': 1.0, 'noSpecialFriends': 1.0}
32+
2023-07-27 18:59:23,313 [INFO] done, 2023-07-27 18:59:23.313776

0 commit comments

Comments
 (0)