Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

无法正常运行ppocrv4_rec_server,其他可以正常运行 #2558

Open
sunzx8 opened this issue Nov 18, 2024 · 1 comment
Open

无法正常运行ppocrv4_rec_server,其他可以正常运行 #2558

sunzx8 opened this issue Nov 18, 2024 · 1 comment
Assignees

Comments

@sunzx8
Copy link

sunzx8 commented Nov 18, 2024


温馨提示:根据社区不完全统计,按照模板提问,可以加快回复和解决问题的速度


环境

  • 【FastDeploy版本】: 说明具体的版本,如fastdeploy-linux-gpu-0.8.0
    • fastdeploy-gpu-python 1.0.7
      fastdeploy-tools 0.0.5
  • 运行指令:
    代码为示例代码:pipeline/multi_thread_process_ocr.py
    运行方法:
    python multi_thread_process_ocr.py --det_model '/home/ubuntu/shawn/ocr-split/ocr_mix/models/ch_PP-OCRv4_det_infer' --cls_model 'ch_ppocr_mobile_v2.0_cls_infer' --rec_model '/home/ubuntu/shawn/ocr-split/ocr_mix/models/ch_PP-OCRv4_rec_server_infer' --rec_label_file '/home/ubuntu/shawn/FastDeploy/tutorials/multi_thread/python/pipeline/ppocr_keys_v1.txt' --image_path '/home/ubuntu/shawn/test_imgs/sda.jpg' --device gpu --use_multi_process True --process_num 3 --backend paddle

问题日志及出现问题的操作流程

  • 附上详细的问题日志有助于快速定位分析
  • 【模型跑不通】
    • ppocrv4的server版本rec 无法运行,其他模型都可以正常运行,但是只要切换到server版本的识别模型就会报错

报错日志:
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/ubuntu/miniconda3/envs/ocr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/ubuntu/miniconda3/envs/ocr/lib/python3.10/multiprocessing/pool.py", line 48, in mapstar
return list(map(*args))
File "/home/ubuntu/shawn/FastDeploy/tutorials/multi_thread/python/pipeline/multi_thread_process_ocr.py", line 219, in process_predict
result = ppocr_v3.predict(im)
File "/home/ubuntu/miniconda3/envs/ocr/lib/python3.10/site-packages/fastdeploy/vision/ocr/ppocr/init.py", line 958, in predict
return self.system_.predict(input_image)
RuntimeError:

Compile Traceback (most recent call last):
File "tools/export_model.py", line 288, in
main()
File "tools/export_model.py", line 284, in main
model, arch_config, save_path, logger, input_shape=input_shape)
File "tools/export_model.py", line 197, in export_single_model
paddle.jit.save(model, save_path)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/jit.py", line 629, in wrapper
func(layer, path, input_spec, **configs)
File "/usr/local/lib/python3.7/dist-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/base.py", line 51, in impl
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/jit.py", line 857, in save
inner_input_spec, with_hook=with_hook)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 528, in concrete_program_specify_input_spec
*desired_input_spec, with_hook=with_hook)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 436, in get_concrete_program
concrete_program, partial_program_layer = self._program_cache[cache_key]
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 801, in getitem
self._caches[item_id] = self._build_once(item)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 790, in _build_once
**cache_key.kwargs)
File "/usr/local/lib/python3.7/dist-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/base.py", line 51, in impl
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 733, in from_func_spec
outputs = static_func(*inputs)
File "/tmp/tmp18grrv7q.py", line 28, in forward
false_fn_1, (x,), (x,), (x,))
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 211, in convert_ifelse
out = _run_py_ifelse(pred, true_fn, false_fn, true_args, false_args)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 257, in _run_py_ifelse
return true_fn(*true_args) if pred else false_fn(*false_args)
File "/workspace/gry/docker/code/luolixin/baidu/paddle/paddle-uapi/uapi/PaddleOCR/ppocr/modeling/architectures/base_model.py", line 86, in forward
x = self.backbone(x)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/tmp/tmparp3ri44.py", line 45, in forward
for_loop_condition_0, for_loop_body_0, [i, __for_loop_var_index_0, x])
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 45, in convert_while_loop
loop_vars = _run_py_while(cond, body, loop_vars)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 59, in _run_py_while
loop_vars = body(*loop_vars)
File "/workspace/gry/docker/code/luolixin/baidu/paddle/paddle-uapi/uapi/PaddleOCR/ppocr/modeling/backbones/rec_hgnet.py", line 252, in forward
x = stage(x)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/workspace/gry/docker/code/luolixin/baidu/paddle/paddle-uapi/uapi/PaddleOCR/ppocr/modeling/backbones/rec_hgnet.py", line 172, in forward
x = self.blocks(x)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/container.py", line 98, in forward
input = layer(input)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/workspace/gry/docker/code/luolixin/baidu/paddle/paddle-uapi/uapi/PaddleOCR/ppocr/modeling/backbones/rec_hgnet.py", line 125, in forward
x = self.att(x)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/workspace/gry/docker/code/luolixin/baidu/paddle/paddle-uapi/uapi/PaddleOCR/ppocr/modeling/backbones/rec_hgnet.py", line 76, in forward
x = self.conv(x)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/nn/layer/conv.py", line 678, in forward
use_cudnn=self._use_cudnn)
File "/usr/local/lib/python3.7/dist-packages/paddle/nn/functional/conv.py", line 169, in _conv_nd
type=op_type, inputs=inputs, outputs=outputs, attrs=attrs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/layer_helper.py", line 44, in append_op
return self.main_program.current_block().append_op(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/framework.py", line 3621, in append_op
attrs=kwargs.get("attrs", None))
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/framework.py", line 2635, in init
for frame in traceback.extract_stack():


C++ Traceback (most recent call last):

0 paddle::AnalysisPredictor::ZeroCopyRun()
1 paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, phi::Place const&)
2 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, phi::Place const&) const
3 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, phi::Place const&, paddle::framework::RuntimeContext*) const
4 void phi::KernelImpl<void ()(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::string const&, std::vector<int, std::allocator > const&, int, std::string const&, std::string const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >), &(void phi::fusion::ConvFusionKernel<float, phi::GPUContext>(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::string const&, std::vector<int, std::allocator > const&, int, std::string const&, std::string const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >))>::KernelCallHelper<paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::string const&, std::vector<int, std::allocator > const&, int, std::string const&, std::string const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >, phi::TypeTag >::Compute<1, 3, 0, 0, phi::GPUContext const, phi::DenseTensor const, phi::DenseTensor const, phi::DenseTensor const>(phi::KernelContext*, phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&)
5 void phi::fusion::ConvFusionKernel<float, phi::GPUContext>(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::string const&, std::vector<int, std::allocator > const&, int, std::string const&, std::string const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >)
6 phi::DnnWorkspaceHandle::RunFunc(std::function<void (void*)> const&, unsigned long)
7 std::_Function_handler<void (void*), phi::fusion::ConvFusionKernel<float, phi::GPUContext>(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::string const&, std::vector<int, std::allocator > const&, int, std::string const&, std::string const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >)::{lambda(void*)#4}>::_M_invoke(std::_Any_data const&, void*&&)
8 phi::enforce::EnforceNotMet::EnforceNotMet(phi::ErrorSummary const&, char const*, int)
9 phi::enforce::GetCurrentTraceBackStringabi:cxx11


Error Message Summary:

ExternalError: CUDNN error(9), CUDNN_STATUS_NOT_SUPPORTED.
[Hint: Please search for the error code(9) on website (https://docs.nvidia.com/deeplearning/cudnn/api/index.html#cudnnStatus_t) to get Nvidia's official solution and advice about CUDNN Error.] (at /home/fastdeploy/develop/paddle_build/v0.0.0/Paddle/paddle/phi/kernels/fusion/gpu/conv_fusion_kernel.cu:611)
[operator < conv2d_fusion > error]
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/ubuntu/shawn/FastDeploy/tutorials/multi_thread/python/pipeline/multi_thread_process_ocr.py", line 250, in
pool.map(process_predict, imgs_list)
File "/home/ubuntu/miniconda3/envs/ocr/lib/python3.10/multiprocessing/pool.py", line 367, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/home/ubuntu/miniconda3/envs/ocr/lib/python3.10/multiprocessing/pool.py", line 774, in get
raise self._value
RuntimeError:

Compile Traceback (most recent call last):
File "tools/export_model.py", line 288, in
main()
File "tools/export_model.py", line 284, in main
model, arch_config, save_path, logger, input_shape=input_shape)
File "tools/export_model.py", line 197, in export_single_model
paddle.jit.save(model, save_path)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/jit.py", line 629, in wrapper
func(layer, path, input_spec, **configs)
File "/usr/local/lib/python3.7/dist-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/base.py", line 51, in impl
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/jit.py", line 857, in save
inner_input_spec, with_hook=with_hook)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 528, in concrete_program_specify_input_spec
*desired_input_spec, with_hook=with_hook)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 436, in get_concrete_program
concrete_program, partial_program_layer = self._program_cache[cache_key]
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 801, in getitem
self._caches[item_id] = self._build_once(item)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 790, in _build_once
**cache_key.kwargs)
File "/usr/local/lib/python3.7/dist-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/base.py", line 51, in impl
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 733, in from_func_spec
outputs = static_func(*inputs)
File "/tmp/tmp18grrv7q.py", line 28, in forward
false_fn_1, (x,), (x,), (x,))
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 211, in convert_ifelse
out = _run_py_ifelse(pred, true_fn, false_fn, true_args, false_args)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 257, in _run_py_ifelse
return true_fn(*true_args) if pred else false_fn(*false_args)
File "/workspace/gry/docker/code/luolixin/baidu/paddle/paddle-uapi/uapi/PaddleOCR/ppocr/modeling/architectures/base_model.py", line 86, in forward
x = self.backbone(x)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/tmp/tmparp3ri44.py", line 45, in forward
for_loop_condition_0, for_loop_body_0, [i, __for_loop_var_index_0, x])
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 45, in convert_while_loop
loop_vars = _run_py_while(cond, body, loop_vars)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 59, in _run_py_while
loop_vars = body(*loop_vars)
File "/workspace/gry/docker/code/luolixin/baidu/paddle/paddle-uapi/uapi/PaddleOCR/ppocr/modeling/backbones/rec_hgnet.py", line 252, in forward
x = stage(x)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/workspace/gry/docker/code/luolixin/baidu/paddle/paddle-uapi/uapi/PaddleOCR/ppocr/modeling/backbones/rec_hgnet.py", line 172, in forward
x = self.blocks(x)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/container.py", line 98, in forward
input = layer(input)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/workspace/gry/docker/code/luolixin/baidu/paddle/paddle-uapi/uapi/PaddleOCR/ppocr/modeling/backbones/rec_hgnet.py", line 125, in forward
x = self.att(x)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/workspace/gry/docker/code/luolixin/baidu/paddle/paddle-uapi/uapi/PaddleOCR/ppocr/modeling/backbones/rec_hgnet.py", line 76, in forward
x = self.conv(x)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 930, in call
return self._dygraph_call_func(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 915, in _dygraph_call_func
outputs = self.forward(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/nn/layer/conv.py", line 678, in forward
use_cudnn=self._use_cudnn)
File "/usr/local/lib/python3.7/dist-packages/paddle/nn/functional/conv.py", line 169, in _conv_nd
type=op_type, inputs=inputs, outputs=outputs, attrs=attrs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/layer_helper.py", line 44, in append_op
return self.main_program.current_block().append_op(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/framework.py", line 3621, in append_op
attrs=kwargs.get("attrs", None))
File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/framework.py", line 2635, in init
for frame in traceback.extract_stack():


C++ Traceback (most recent call last):

0 paddle::AnalysisPredictor::ZeroCopyRun()
1 paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, phi::Place const&)
2 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, phi::Place const&) const
3 paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, phi::Place const&, paddle::framework::RuntimeContext*) const
4 void phi::KernelImpl<void ()(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::string const&, std::vector<int, std::allocator > const&, int, std::string const&, std::string const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >), &(void phi::fusion::ConvFusionKernel<float, phi::GPUContext>(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::string const&, std::vector<int, std::allocator > const&, int, std::string const&, std::string const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >))>::KernelCallHelper<paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::string const&, std::vector<int, std::allocator > const&, int, std::string const&, std::string const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >, phi::TypeTag >::Compute<1, 3, 0, 0, phi::GPUContext const, phi::DenseTensor const, phi::DenseTensor const, phi::DenseTensor const>(phi::KernelContext*, phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&)
5 void phi::fusion::ConvFusionKernel<float, phi::GPUContext>(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::string const&, std::vector<int, std::allocator > const&, int, std::string const&, std::string const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >)
6 phi::DnnWorkspaceHandle::RunFunc(std::function<void (void*)> const&, unsigned long)
7 std::_Function_handler<void (void*), phi::fusion::ConvFusionKernel<float, phi::GPUContext>(phi::GPUContext const&, phi::DenseTensor const&, phi::DenseTensor const&, phi::DenseTensor const&, paddle::optionalphi::DenseTensor const&, std::vector<int, std::allocator > const&, std::vector<int, std::allocator > const&, std::string const&, std::vector<int, std::allocator > const&, int, std::string const&, std::string const&, bool, std::vector<int, std::allocator > const&, int, phi::DenseTensor*, std::vector<phi::DenseTensor*, std::allocatorphi::DenseTensor* >)::{lambda(void*)#4}>::_M_invoke(std::_Any_data const&, void*&&)
8 phi::enforce::EnforceNotMet::EnforceNotMet(phi::ErrorSummary const&, char const*, int)
9 phi::enforce::GetCurrentTraceBackStringabi:cxx11


Error Message Summary:

ExternalError: CUDNN error(9), CUDNN_STATUS_NOT_SUPPORTED.
[Hint: Please search for the error code(9) on website (https://docs.nvidia.com/deeplearning/cudnn/api/index.html#cudnnStatus_t) to get Nvidia's official solution and advice about CUDNN Error.] (at /home/fastdeploy/develop/paddle_build/v0.0.0/Paddle/paddle/phi/kernels/fusion/gpu/conv_fusion_kernel.cu:611)
[operator < conv2d_fusion > error]

@Alex37882388
Copy link

重装cudnn v8.6.0,包治病

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants