Advanced Usage

g4f advanced usage

---

In Basic Usage we already discussed, how to use g4f to generate Completions.

tldr: we import Client from g4f.client and create a Client to initialise g4f. We then use the client.chat.completions.create method to generate completions. We can also stream the completion, by setting stream=True.

from g4f.client import Client

client = Client()

chat_completion = client.chat.completions.create(model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Hello"}], stream=True)

for completion in chat_completion:
    print(completion.choices[0].delta.content or "", end="", flush=True)

The completions.create(...) function

Lets now dive into more advanced usage of g4f. The create function has various additional parameters that can be used to customise the completions generated.

create(...) additional parameters

proxy

string

default: "None"

The proxy to use for the API request, more on proxies can be found here.

chat_completion = client.chat.completions.create(...
    proxy="http://user:password@host:port")

response_format

dict

default: "None"

An object specifying the format that the model must output. Compatible with advanced llm’s.
Setting to { "type": "json_object" } enables JSON mode, which guarantees the message the model generates is valid JSON.

Important: when using JSON mode, you must also instruct the model to produce JSON yourself via a system or user message.

max_tokens

integer

default: "None"

The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length.

stop

list[string] | string

default: "None"

Up to 4 sequences where the API will stop generating further tokens.

api_key

string

The API key to use for the request. Defaults to None.

ignored

list[string]

A list of strings to ignore in the response. Defaults to None.

ignore_working

bool

Whether to ignore the working status of the provider. Defaults to False.

ignore_stream

bool

Whether to ignore the stream status of the provider. Defaults to False.

You can use kwargs to pass additional parameters like temperature, they are not guaranteed to work, and depend on how much customisation a certain Provider exposes in his Api.

kwargs

Any

Additional keyword arguments to pass to the API request.

Provider rotating in g4f

g4f can be a bit buggy sometimes and cause you some trouble. You can create your own providers list and pass it with RetryProvider to Client to make g4f more stable.

from g4f.client   import Client
from g4f.Provider import (
  ProviderA,
  ProviderB,
  ProviderC,
  RetryProvider
)

import g4f

client = Client(
    provider = RetryProvider([
        ProviderA,
        ProviderB,
        ProviderC
    ])
)

...

RetryProvider will try to use the first provider in the list, if it fails, it will try the next one, and so on. Ensuring a higher chance of success. If you wish to use a single provider with retry use it like so:

from g4f.client   import Client
from g4f.Provider import OpenaiChat, RetryProvider
import g4f

client = Client(
    provider = RetryProvider([OpenaiChat],
                             single_provider_retry=True, max_retries=5)
)

This works well if there is a very good provider that fails sometimes.