- Big Data News Weekly
- Posts
- Sparkle: Standardizing Modular ETL
Sparkle: Standardizing Modular ETL
🦾Plus: 🧠 A New AGI Test beats Smartest AI Models
Hey folks! Let’s get into Big Data and AI craziness…
In today's edition: What's Shaping the Future of Data?
🐍20 Tools for writing Better Python Code
💡The Software Engineering Identity Crisis
🤖KBLaM: Knowledge Base augmented Language Model
🌍UAE pledges $1.4 trillion investment in US
🌜 Reve’s new leading image model
💻 AI Tutorial:Create custom AI videos to boost engagement
🤖 AI Tools and Data Tools to checkout
In 2023, the Uber Data platform migrated all batch workloads to Apache Spark™-based computation. Around 20,000+ critical pipelines and datasets are used to power the batch workloads and more than 3,000+ engineers are responsible for creating pipelines and owning datasets.
Instead of trying to predict which party will win, and where to invest afterwards, why not invest in an ‘election-proof’ alternative asset? The sector is currently in a softer cycle, but over the last seven elections (1995-2023) blue-chip contemporary art has outpaced the S&P 500 by 64% even despite the recent dip, regardless of the victors, and we have conviction it will rebound to these levels long-term.
Now, thanks to Masterworks’ art investing platform, you can easily diversify into this asset class without needing millions or art expertise, alongside 65,000+ other art investors. From their 23 exits so far, Masterworks investors have realized representative annualized net returns like +17.6%, +17.8%, and +21.5% (among assets held longer than one year), even despite a recent dip in the art market.*
Python is a powerful and versatile programming language, but writing clean, efficient, and error-free code can be challenging. Thankfully, there are numerous tools available to help developers enhance their Python coding skills. Below are 20 essential tools that can improve your Python development experience.
Many software engineers want to build things, not manage or oversee things - but that identity is being challenged with the introduction of AI coding assistants.
In-context learning, while simple, becomes computationally inefficient with large knowledge bases. To overcome these limitations, Microsoft introduces KBLaM (Knowledge Base-Augmented Language Model), a novel approach that integrates structured knowledge directly into LLMs using a scalable, efficient mechanism called rectangular attention.
Unlock the potential of your growing small business with HubSpot's Starter Customer Platform. For a discounted price of just $20/month, unlock access to the Starter edition of HubSpot’s six core products for marketing, sales, and customer service – powered by HubSpot’s Smart CRM – all for the price of one. Built for businesses like yours, HubSpot Starter has all the essential tools you need to scale.
Click here to claim your small business bundle and start growing your business today!
👨💻 Data Tools, Libraries
ingestr (GitHub Repo)
ingestr is a command-line tool that can copy data between databases with a single command. It can copy data from any source to any destination without any code.
Litestar (Website)
Litestar is a lightweight and flexible ASGI framework for building performant APIs.
FileQL
A tool that allow you to run SQL-like query on local files instead of database files using the GitQL SDK.
AI News:
The Arc Prize Foundation, co-founded by AI researcher François Chollet, just dropped a new test for general intelligence in AI: ARC-AGI-2. And guess what?
Here’s how the top models performed:
OpenAI’s o1-pro & DeepSeek R1: 1% to 1.3%
GPT-4.5, Claude 3.7 Sonnet, Gemini 2.0 Flash: ~1%
OpenAI’s o3, which crushed the older ARC-AGI-1 with 75.7%, scored just 4% on ARC-AGI-2 — using $200 per task 😳
In comparison, human panels averaged 60% accuracy
The United Arab Emirates has pledged $1.4 trillion over the next decade to boost its investments in U.S. sectors such as AI infrastructure, semiconductors, energy, and manufacturing, building on its existing $1 trillion already invested.
Reve just emerged from stealth with Reve Image 1.0, a new text-to-image AI model that topped global rankings with the codename “Halfmoon” over the last week—showcasing exceptional prompt accuracy, text rendering, and image quality. The model claimed the #1 position in Artificial Analysis' Image Arena, outperforming rivals like Google's Imagen 3, Midjourney v6.1, and Recraft V3.
Design a custom website with Squarespace's professionally curated layout and styling options designed to sell anything. Start with a flexible designer template or build your own, then customize to fit your style using our drag-and-drop website tool.
Google's search engine is one of the most profitable technologies ever developed, and there's little evidence that this is changing, but the company is acting with urgency, focusing on generative AI efforts. Multiple independent web publishers say their traffic has been falling as Google's AI overviews present information directly on Google's own results pages.
AI Tutorial
🎬 Create custom AI videos to boost engagement
In this tutorial, you will learn how to use Synthesia's AI video platform to create personalized videos using AI avatars and add them to your emails to increase response rates.
Step-by-step:
Create a free Synthesia account, select a template, and add your script and on-screen text to create a basic video.
Add personalization variables using double curly brackets in both your script and on-screen text, then convert to a template.
Use your template to create individual videos by filling in the variable fields for each recipient.
Copy video thumbnails as GIFs directly into your emails and add call-to-action buttons to drive conversions.
Celebrate the season by learning a new language with Babbel. Developed by over 150 expert linguists, Babbel’s premium platform is scientifically proven to help you learn faster. With award-winning lessons, immersive podcasts, addictive games, and their new AI Conversation Partner, you can start speaking a new language in as little as 3 weeks. Plus, Big Data News Weekly readers can get 55% off with this exclusive link.
🔥Top AI tools to increase productivity:
Udioai - An app for music creation and sharing that allows you to generate amazing music
ChatSweetie chatbot, you can chat online with your chosen virtual character anytime
Exploding Insights - The #1 Market Research Tool. Finding the right idea used to be hard
StudyGPT is your personal AI study assistant！
Suno-top, your ultimate companion for music creation and sharing!
Augment Code - The first AI coding assistant built for professional software engineers and large codebases.
X Headshot is an AI headshot generator that turns your selfies into professional AI headshots.
View our database of all the best AI tools for your needs: aitoolsup.com
Have cool resources to share? Submit AI tool
A.I. Generated Image of the Day
👀 You’re what you eat!
Recommended reading
SPONSOR US
Get your product in front of Big Data & AI enthusiasts
Our newsletter is read by thousands of tech professionals, investors, engineers, managers, and business owners around the world.
Interested in Sponsoring the Big Data News Weekly Newsletter?Get in touch today
What did you think of today's email?Your feedback helps me create better emails for you! |