Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to convert large model from Keras #640

Open
emerzon opened this issue Aug 6, 2023 · 0 comments
Open

Unable to convert large model from Keras #640

emerzon opened this issue Aug 6, 2023 · 0 comments

Comments

@emerzon
Copy link

emerzon commented Aug 6, 2023

While trying to convert from Keras, I get the following error:

Traceback (most recent call last):
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tf2onnx\tf_loader.py", line 218, in from_trackable
    frozen_graph = from_function(concrete_func, inputs, outputs, large_model)
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tf2onnx\tf_loader.py", line 277, in from_function
    raise e
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tf2onnx\tf_loader.py", line 270, in from_function
    frozen_func = convert_variables_to_constants_v2(func, lower_control_flow=False, aggressive_inlining=True)
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\framework\convert_to_constants.py", line 1162, in convert_variables_to_constants_v2
    return _construct_concrete_function(func, output_graph_def,
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\framework\convert_to_constants.py", line 1087, in _construct_concrete_function
    new_func = wrap_function.function_from_graph_def(output_graph_def,
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\eager\wrap_function.py", line 655, in function_from_graph_def
    wrapped_import = wrap_function(_imports_graph_def, [])
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\eager\wrap_function.py", line 619, in wrap_function
    func_graph.func_graph_from_py_func(
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\framework\func_graph.py", line 1247, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\eager\wrap_function.py", line 83, in __call__
    return self.call_with_variable_creator_scope(self._fn)(*args, **kwargs)
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\eager\wrap_function.py", line 89, in wrapped
    return fn(*args, **kwargs)
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\eager\wrap_function.py", line 649, in _imports_graph_def
    importer.import_graph_def(graph_def, name="")
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\util\deprecation.py", line 561, in new_func
    return func(*args, **kwargs)
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\framework\importer.py", line 403, in import_graph_def
    return _import_graph_def_internal(
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tensorflow\python\framework\importer.py", line 497, in _import_graph_def_internal
    with c_api_util.tf_buffer(graph_def.SerializeToString()) as serialized:
ValueError: Message tensorflow.GraphDef exceeds maximum protobuf size of 2GB: 4797467584

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\ui\convert.py", line 8, in <module>
    onnx_model = onnxmltools.convert_keras(model)
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\onnxmltools\convert\main.py", line 84, in convert_keras
    model_proto, external_tensor_storage = tf2onnx.convert.from_keras(
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tf2onnx\convert.py", line 497, in from_keras
    tf_loader.from_trackable(model, concrete_func, input_names, output_names, large_model)
  File "C:\Users\emers\OneDrive\Documents\Code\ml4\.conda\lib\site-packages\tf2onnx\tf_loader.py", line 221, in from_trackable
    raise ValueError(err_large_model)
ValueError: model exceeds maximum protobuf size of 2GB. Try setting large_model.

However, it's not possible to set the large_model=True parameter for tf2onnx - Which is always passed as false.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant