Timeout Error

[mjshanno]

Hi all,
I'm trying to bring some website json text onto my computer. My code below is timing out for some reason. When the json text I am downloading is smaller, I have no issues. Increasing the timeout time period does not seem to solve the problem.

I also have no problems running this code in selenium (not selenium-driverless), so I think it has to do with some setting in selenium-driverless or asyncio. Any help would be greatly appreciated. Thanks!

The code works when I get less json text from here: 'https://eu-offering-api.kambicdn.com/offering/v2018/rsiuspa/listView/football/all/all/all/in-play.json?market=US&includeParticipants=true&useCombined=true&lang=en_US'

from selenium_driverless import webdriver
from selenium_driverless.types.by import By
import asyncio


async def events_pull_kb():
    options = webdriver.ChromeOptions()
    async with webdriver.Chrome(options=options) as driver:
        await driver.get('https://eu-offering-api.kambicdn.com/offering/v2018/rsiuspa/listView/football/all/all/all/matches.json?market=US&includeParticipants=true&useCombined=true&lang=en_US')
        await driver.sleep(0.5)
        kb_event_text = (await driver.find_element(by=By.XPATH, value='/html/body')).text
       
        return await kb_event_text

kb_events = asyncio.run(events_pull_kb())

[ttlns]

@mjshanno
Surely, I can give it a test.
probably max websocket message size.

[mjshanno]

@kaliiiiiiiiii thanks for looking into this. I went into \websockets\sync\server.py and increased the max_size, but that doesn't seem to resolve the issue. I'm not sure I did the right thing to increase the websocket message size though because I'm still learning how this code works.

[ttlns]

uh probably in /cdp-socket/socket more likely

[mjshanno]

unfortunately, I haven't been able to figure out a solution. Let me know if you find one

[ttlns]

@mjshanno Can you maybe provide the full stack of your error? (make sure to execute pip install --upgrade selenium-driverless before )

[mjshanno]

Yes, selenium-driverless is now updated and full stack below. Also, the initial json link I provided has shrunk in size, so it is now not having an issue. If you replace that url with this bigger json, the error will happen: 'https://api.americanwagering.com/regions/us/locations/pa/brands/czr/sb/v3/events/be938c7d-8f2a-4189-9139-b9e2bdad9d74'

Traceback (most recent call last):

File ~\anaconda3\Lib\asyncio\tasks.py:490 in wait_for
return fut.result()

File ~\anaconda3\Lib\asyncio\futures.py:198 in result
raise exc

CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File ~\anaconda3\Lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
exec(code, globals, locals)

File c:\users\micha\documents\get_odds_py\code_support\driverless\untitled1.py:18
kb_events = asyncio.run(events_pull_kb())

File ~\anaconda3\Lib\site-packages\nest_asyncio.py:31 in run
return loop.run_until_complete(task)

File ~\anaconda3\Lib\site-packages\nest_asyncio.py:99 in run_until_complete
return f.result()

File ~\anaconda3\Lib\asyncio\futures.py:203 in result
raise self._exception.with_traceback(self._exception_tb)

File ~\anaconda3\Lib\asyncio\tasks.py:267 in __step
result = coro.send(None)

File c:\users\micha\documents\get_odds_py\code_support\driverless\untitled1.py:16 in events_pull_kb
return await kb_event_text

File ~\anaconda3\Lib\site-packages\selenium_driverless\types\webelement.py:355 in text
return await self.get_property("textContent")

File ~\anaconda3\Lib\site-packages\selenium_driverless\types\webelement.py:344 in get_property
return await self.execute_script(f"return obj[arguments[0]]", name)

File ~\anaconda3\Lib\site-packages\selenium_driverless\types\webelement.py:757 in execute_script
return await self.exec(script, *args, max_depth=max_depth, serialization=serialization,

File ~\anaconda3\Lib\site-packages\selenium_driverless\types\deserialize.py:196 in exec
res = await self.exec_raw(script, *args, max_depth=max_depth,

File ~\anaconda3\Lib\site-packages\selenium_driverless\types\deserialize.py:148 in exec_raw
raise e

File ~\anaconda3\Lib\site-packages\selenium_driverless\types\deserialize.py:140 in exec_raw
res = await self.target.execute_cdp_cmd("Runtime.callFunctionOn", args, timeout=timeout)

File ~\anaconda3\Lib\site-packages\selenium_driverless\types\target.py:743 in execute_cdp_cmd
result = await self.socket.exec(method=cmd, params=cmd_args, timeout=timeout)

File ~\anaconda3\Lib\site-packages\cdp_socket\socket.py:76 in exec
raise e

File ~\anaconda3\Lib\site-packages\cdp_socket\socket.py:72 in exec
return await asyncio.wait_for(self._responses[_id], timeout=timeout)

File ~\anaconda3\Lib\asyncio\tasks.py:492 in wait_for
raise exceptions.TimeoutError() from exc

TimeoutError

[ttlns]

@mjshanno Turns out this actually is the websocket max-size its fault.
Wonder why it didn't manage to report that correctly tho.
Anyways, the following script worked just fine for me (maybe the ws_max_size is a bit to high, no idea tbh):

from selenium_driverless import webdriver
from selenium_driverless.types.by import By
import asyncio


async def events_pull_kb():
    options = webdriver.ChromeOptions()
    async with webdriver.Chrome(options=options, max_ws_size=2**50) as driver:
        await driver.get(
            'https://api.americanwagering.com/regions/us/locations/pa/brands/czr/sb/v3/events/be938c7d-8f2a-4189-9139-b9e2bdad9d74')
        elem = await driver.find_element(by=By.XPATH, value='/html/body')
        kb_event_text = await elem.text
        return kb_event_text


kb_events = asyncio.run(events_pull_kb())
print(kb_events)

feel free to close this dicussion in case your issue is resolved now:)

[mjshanno]

That fixes it, thank you so much!