Question (or problem) about logs and subsequent difficulties in builds

[dbollini]

Hello everyone!

Here Diego from Argentina (adhoc.com.ar), I'm not sure if it's an issue of the application or it's a problem of our implementation so first I consult here. I'm not sure if this section is active but let's go.

Maybe we had a problem that was fixed by pulling the latest runbot 15 (because indeed certain messages we were seeing with GHOST for example disappeared and certain runbot commits would give that indication). But maybe when updating runbot we brought up another issue regarding the logs as there were several changes like [IMP] runbot: add an option on step to disable logdb, [FIX] runbot: remove hardcoded runbot_logs or [IMP] runbot: queue build logs in a local database.

We detected that the builds were not advancing in the respective steps. Reviewing the runbot code and logs, we noticed that we were accumulating too many logs in ir_logging.

The first time we detected it there were millions (I don't remember the exact number but it was a lot), but recently we saw these values generating problems:

# select count(*) from ir_logging;
count = 1656
# select max(id) from ir_logging;
max = 645195

Our (non) solution was:
TRUNCATE ir_logging RESTART IDENTITY;

Has it happened to you too? I know it's a bit vague and context-less, but any questions and observations are welcome. Thanks!