Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GigaChat API #2495

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Conversation

seldereyy
Copy link

@seldereyy seldereyy commented Nov 15, 2024

This PR is an update for #1996 .

This PR adds the GigaChat models API. GigaChat models are proprietary, so the log-likelihood methods are not available.
Two authorization methods are available: using an access token or API credentials.
If you have an access token, set the GIGACHAT_CREDENTIALS environmental variable with its value and use base_url for your token.
For authorization through API credentials, set the following environmental variables:

  • GIGACHAT_RQUID - Unique identification request. Complies with the uuid4 format. The value must match the regular expression ([0-9a-fA-F-]{36}).
  • GIGACHAT_TOKEN - Client Secret. Credential for the GigaChat API.
  • GIGACHAT_SCOPE - Type of your API. Possible values: ["GIGACHAT_API_PERS", "GIGACHAT_API_B2B", "GIGACHAT_API_CORP"]. The default value is "GIGACHAT_API_PERS".

You also need to set base_url and auth_url in the model_args parameter.

  • base_url - URL for chat completion.
  • auth_url - URL for obtaining the authorization token.

Choose a model from the available options: GigaChat, GigaChat-Plus, GigaChat-Pro, GigaChat-Max.

Here is a usage example for the GSM8K benchmark with API authorization:

export GIGACHAT_TOKEN="XXX"
export GIGACHAT_RQUID="XXX"

lm_eval --model gigachat_llms \
    --model_args model="MODEL_NAME",base_url="BASE_URL",auth_url="AUTH_URL" \
    --tasks gsm8k \
    --device cpu \
    --write_out \
    --apply_chat_template \
    --log_samples \
    --output_path 'OUTPUT_PATH'

Example for access_token authorization:

export GIGACHAT_CREDENTIALS="XXX"

lm_eval --model gigachat_llms \
    --model_args model="MODEL_NAME",base_url="BASE_URL" \
    --tasks gsm8k \
    --device cpu \
    --write_out \
    --apply_chat_template \
    --log_samples \
    --output_path 'OUTPUT_PATH'

Some technical details:

  • We use the property decorator (not cached_property) for the api_key and header methods because we need to issue a new access_token every 30 minutes.
  • The GigaChat API does not have a "do sample" option. We removed it and use the top_p parameter instead.

@seldereyy seldereyy mentioned this pull request Nov 15, 2024
@seldereyy seldereyy marked this pull request as draft November 15, 2024 14:03
@seldereyy seldereyy marked this pull request as ready for review November 27, 2024 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant