v1.2.24

Sling v1.2.24 (2024-11-26T23:47:50Z)

Fix Prometheus basic auth

v1.2.23

Sling v1.2.23

Major Changes

1. Removed Iterate Flag

   • Removed the iterate functionality from CLI run command
   • Simplified run logic by removing iteration loop

2. Database Improvements

   • Added better context cancellation handling in database streaming
   • Enhanced SQLite binary download URLs
   • Improved SQL Server connection string handling
   • Updated Oracle SQLLoader column handling for large strings
   • Added support for unsigned integers in MySQL/MariaDB type mappings

3. SFTP Enhancements

   • Updated SFTP file creation to handle SSH_FX_OP_UNSUPPORTED errors
   • Added fallback to OpenFile when Create is not supported

4. Snowflake Updates

   • Renamed internalStage property to internal_stage
   • Added schema usage before operations
   • Fixed stage folder path construction

Minor Changes

1. URL Fixes

   • Fixed homepage URLs in goreleaser configs
   • Updated various CDN URLs to use new domain

2. UI/Documentation

   • Updated Discord badge in README
   • Added execution ID to metadata tracking
   • Improved error messages and logging

3. Dependencies

   • Updated various dependencies, including github.com/flarco/g
   • Updated SFTP package to newer version

4. Configuration

   • Added support for extra BCP arguments in SQL Server

Bug Fixes

• Fixed byte counting in CSV writing
• Improved error handling in various database operations
• Fixed constraint failure handling
• Enhanced context cancellation checks in database operations

v1.2.22

Sling v1.2.22

1. Direct Insert Support

• Added new SLING_DIRECT_INSERT environment variable to enable direct table inserts
• Implemented writeToDbDirectly() function to bypass temp table creation
• Added new test cases for direct insert functionality

2. File Path Improvements

• Added CleanTableName() function to sanitize table names in temp file paths
• Updated temp file paths across multiple database connectors to use cleaned table names
• Made temp file paths more consistent and descriptive

3. Stream Configuration Updates

• Refactored stream configuration handling to use structured configs instead of maps
• Added LoaderStreamConfig() for standardized loader configurations
• Improved compression and datetime format handling

4. Database Connector Enhancements

• Added trusted connection support for SQL Server
• Updated Prometheus client authentication
• Improved error handling and transaction management
• Enhanced column casing support for upserts

v1.2.21

Sling v1.2.21

Main Changes

1. Dependency Updates

   • Updated several Go module dependencies, including golang.org/x/crypto, golang.org/x/exp, golang.org/x/net, golang.org/x/sys, golang.org/x/term, golang.org/x/text, golang.org/x/time, golang.org/x/tools, and various Google APIs.
   • Updated gorm.io/gorm from v1.25.9 to v1.25.10.
   • Updated github.com/trinodb/trino-go-client from v0.316.0 to v0.318.0.

2. GitHub Actions Updates

   • Updated the Infisical action from v2 to v3 across multiple workflow files.
   • Modified environment variable handling in workflows.

3. Code Improvements

   • Enhanced error handling and logging in various parts of the codebase.
   • Improved configuration parsing and handling.
   • Updated CLI command processing and output formatting.

4. Documentation Updates

   • Significantly revised the README.md, particularly the "Contributing" section.
   • Added more detailed guidelines for branch naming, testing, and pull request processes.

5. Testing Enhancements

   • Added new test files and updated existing test suites.
   • Improved test configuration and execution instructions.

v1.2.20

Sling v1.2.20

Bug Fixes

• Improved error handling in MergeDataflow function
  • Now properly closes datastream on error
• Fixed potential issue in MakeShaper function
  • Now uses Names(true) to get column names, ensuring consistent behavior

v1.2.19

Sling v1.2.19 (2024-09-22T20:03:08Z)

Summary of Changes

Configuration and Templates

• Updated various database templates (e.g., Azure SQL, BigQuery, Clickhouse, DuckDB, MariaDB, MySQL, Oracle, Postgres, Redshift, Snowflake, SQLite, SQL Server, StarRocks, Trino) to include:

  • Added schema_name and is_view columns to table and view queries
  • Modified queries to support optional schema filtering
  • Standardized ordering of results

• Updated DuckDB template:

  • Added stream scanner functions for various file formats
  • Added timestamp layout variables

• Updated Motherduck template:

  • Added scan functions for Iceberg, Delta, and Parquet formats

Core Functionality

• Enhanced environment variable handling in env.go
• Improved error handling and panic recovery in config unmarshalling
• Added support for SQL-based streaming in file sources
• Enhanced incremental value handling for database sources
• Improved wildcard processing for replication configs
• Added dataset output option for file targets

Build and CI Scripts

• Added connection discovery step to build scripts
• Created new preparation scripts for Linux, macOS, and Windows

Testing

• Updated test scripts to use a centralized CLI test

Dependencies

• Updated various Go module dependencies

v1.2.18

Sling v1.2.18 (2024-08-30T08:29:24Z)

Changelog

• Fix sling conns test

v1.2.17

Sling v1.2.17 (2024-08-28T16:39:52Z)

Changelog

• Fix Setting & GetMachineID
• Add home/sling in Dockerfiles
• Update caching and logging

v1.2.16

Sling v1.2.16 (2024-08-27T21:25:58Z)

Changelog Summary

Docker and CI/CD

• Updated Dockerfiles and CI workflows
• Improved build processes for dev and production
• Added self-hosted Mac to build-dev workflow

Database Connections and Operations

• Enhanced connection handling for various databases (DuckDB, Snowflake, Oracle)
• Improved timestamp and timezone handling
• Added support for Delta and Iceberg formats
• Enhanced file system operations and caching

Testing and Error Handling

• Added and updated various tests
• Improved error handling and logging
• Added panic recovery in Parquet operations

Features and Improvements

• Added column constraints and improved column ordering
• Enhanced replication and incremental mode functionality
• Improved wildcard matching and folder glob operations
• Added support for Azure service principal keys
• Implemented connection objects and entries

Code Refactoring and Cleanup

• Moved and reorganized various functions and types
• Cleaned up and updated dependencies (go.mod)
• Improved code structure and naming conventions

Security and Configuration

• Updated SSH/SFTP key exchanges
• Improved secrets handling in CI/CD
• Updated gitignore and other configuration files

v1.2.15

Sling v1.2.15

Major Enhancements

• Performance Improvements

  • Enhanced CastVal function
  • Improved setGetMetadata for StarRocks
  • Updated fs.GetRefTs to handle timestamp values
  • Improved Replace0x00 function

• New Features

  • Added SSH tunnel support for FTP, S3 (MinIO), and SFTP
  • Implemented KEEP_TEMP_FILES option for BigQuery and StarRocks
  • Added EnvFileConnectionEntries and LoadSlingEnvFileBody
  • Introduced store.Settings
  • Implemented primary key and index table-key logic
  • Added column coercion for BigQuery, Snowflake & Redshift sources

Bug Fixes

• Resolved issues with ClickHouse tests and DDL with primary key
• Fixed blank value logic when using JSON
• Addressed ds.SetFields functionality
• Fixed MotherDuck checksum_boolean issue

Database-Specific Updates

• ClickHouse: Improved DDL handling with primary key
• BigQuery: Allow partitioning with custom SQL
• Snowflake, BigQuery, DuckDB, Redshift: Added dummy select for create_index & drop_index
• Oracle: Updated client handling
• Microsoft SQL Server: Upgraded driver

Configuration and Logging

• Enhanced logging throughout the application
• Improved DefaultStreamConfig to preserve transforms
• Updated SQL logging to retain full text

Dependency Updates

• Multiple updates to github.com/flarco/g
• Upgraded github.com/microsoft/go-mssqldb

Miscellaneous

• Expanded allowed glob pattern
• Improved sling update process
• Added warning for large single parquet file writes
• Updated README and documentation