Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hotfix for problem with new BQ loader with shorter col names #42

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

tv2-tjebsen
Copy link

@tv2-tjebsen tv2-tjebsen commented Jul 3, 2024

Description & motivation

When we have implemented the new BQ loader that has similar style to other warehouses with cols named like my_schema_1 and not my_schema_1_0_0, the built in logic of normalize broke. This is a simple fix to the problem but should probably be handled better to account for all changes. We have further problems as well but most of these can be fixed with custom scripts etc.

Please chime in with comments or other suggestions. We really like the normalize package

Checklist

  • I have verified that these changes work locally
  • [N/A] I have updated the README.md (if applicable)
  • [N/A] I have added tests & descriptions to my models (and macros if applicable)
  • [N/A] I have raised a documentation PR if applicable (Link here if required)

@tv2-tjebsen tv2-tjebsen requested a review from a team as a code owner July 3, 2024 20:01
@snowplowcla snowplowcla added the cla:yes [Auto generated] Snowplow Contributor License Agreement has been signed. label Jul 3, 2024
@agnessnowplow
Copy link
Contributor

Hi @tv2-tjebsen, thanks for raising this! We understand the issue and are investigating to come up with a fix on this.

@tv2-tjebsen
Copy link
Author

Are there any news on this? We keep hitting cases where this is a problem, as many of our events start with the same prefixes, eg. ..._video_start and ..._video_end that gets values from each other when the different event happens

@agnessnowplow
Copy link
Contributor

Hi @tv2-tjebsen , sorry for the wait, I've checked with the product team and the plan is to add this in for our next sprint which means it should be released in roughly two weeks. In the meantime you could temporarily overwrite it in your dbt project as a macro overwrite with the same name (as it is in a dispatch macro).

@bassrock
Copy link

@agnessnowplow is there any update on releasing this?

@agnessnowplow
Copy link
Contributor

Hi @bassrock, apologies for the silence on this, we are working on release v0.4.0 at the moment which will tackle this topic as well. The release is expected within the coming week or two.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla:yes [Auto generated] Snowplow Contributor License Agreement has been signed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants