Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Profiling] Help needed with Profiling CT2 on both CUDA and CPU #1734

Open
prasad-pr-20 opened this issue Jun 28, 2024 · 1 comment · May be fixed by #1739
Open

[Profiling] Help needed with Profiling CT2 on both CUDA and CPU #1734

prasad-pr-20 opened this issue Jun 28, 2024 · 1 comment · May be fixed by #1739

Comments

@prasad-pr-20
Copy link

Hi All,

I'm trying to profile the ct2 model on both CPU and CUDA, but I'm unable to see any data output when using the '-DENABLE_PROFILING=ONflag inCMakeLists.txt`.

Specifically, I'm trying to profile the transformers module on both CUDA and CPU. I have attempted to wrap the init_profiling and dump_profiling functions around the model, but I still don't see any profiling outputs on the cout stream.

Could anyone provide guidance on this issue or point me to the relevant documentation or examples? Any help would be greatly appreciated.

Thanks in advance!
@minhthuc2502
Copy link
Collaborator

You can try with:

  • Compile code with -DENABLE_PROFILING=ON
  • In main.cpp:
ctranslate2::init_profiling(ctranslate2::Device::CUDA, 1);
Load model, generate/translate...
ctranslate2::dump_profiling(std::cerr);

It should work like this. You can see this translator as an example.

@yehudaorel yehudaorel linked a pull request Jul 8, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Implement python bindings for profiler yehudaorel/CTranslate2
2 participants
@minhthuc2502 @prasad-pr-20