@sriwworldofcoding.bsky.socialBreaking a production pipeline mid-run is scary.
Rerunning it without checking Delta Lake idempotency is a career hazard. 💀
Here is the exact 4-step protocol senior Azure Data Engineers use to recover cleanly without corrupting downstream tables: 🧵👇
@numberchecker.bsky.socialMyth: Bulk list checking is just a final step to clean up your CRM. Reality: It is a vital schema-validation gateway. Using it pre-ingestion prevents data bottlenecks and ensures your pipeline stays healthy from the start. https://numberchecker.ai/?utm_source=bsky #DataEngineering #CRM
@rajeshdataai.bsky.socialFull Post: www.instagram.com/p/DdHG6YmE0P...
.
.
#SQL #interview #DataScience #DataAnalytics #ArtificialIntelligence #data #BusinessAnalyst #DataAnalyst #Database #DataEngineering #Tech #RajeshUmrao
@rajeshdataai.bsky.social
@rajeshdataai.bsky.socialFull Post: www.instagram.com/p/DdJrxi0E6I...
.
.
#SQL #interview #DataScience #DataAnalytics #ArtificialIntelligence #data #BusinessAnalyst #DataAnalyst #Database #DataEngineering #Tech #RajeshUmrao
@rajeshdataai.bsky.social
@sriwworldofcoding.bsky.social💼 Role: Senior Data Engineer
🏢 Company: ThinkWise Consulting LLP
📍 Location: Hyderabad, Telangana, IN
🔗 artha.link/@sriwworldof...
#DataEngineering #PySpark #AWS #Databricks #CloudComputing #ETL #BigData
@darryl-ruggles.cloud#DuckDB #AWSLambda #ApacheIceberg #S3Tables #DataEngineering
DuckDB has been in the news lately with AWS acquiring DuckLabs. Many people have been using DuckDB for a long time and really appreciate its performance. I like reading examples of tools like this, and there is
@sergiocuellar.bsky.socialAWS Glue 6.0 boasts 30% lower pricing and full Apache Iceberg v3 support. Features include VARIANT data type for faster semi-structured data querying and Spark 4.1 updates. #AWSGlue #ApacheIceberg #DataEngineering 🚀
@powerbitips.bsky.social561 - Snowflake keeps getting easier to work with inside Power BI. Databricks keeps grinding against it. The friction is pushing teams into a split that changes who owns which half of the stack.
@powerbitips.bsky.social561 - A client asked for a data warehouse a couple years back. Nothing was in SQL. It was all SharePoint lists. That conversation had to go somewhere before the word warehouse came up again.
@abhayit2000.bsky.social🚀 Data engineering is becoming a strategic business capability.
AI-native platforms, lakehouses, real-time processing, #DataOps, observability, #DataMesh, LLMs, governance & more are shaping the future.
Explore 15 key trends. 👇
@walookup.bsky.socialWhy does our batch API return the whole set or nothing at all? It’s not a limitation—it’s an atomic safeguard for your database to ensure pipeline consistency. How do you handle synchronization in your data workflows? See the docs at https://walookup.com/?utm_source=bsky #DataEngineering #API
@sriwworldofcoding.bsky.socialLevel up your data engineering architecture game:
📖 Read the full Medium guide:
medium.com/@sriwworldof...
▶️ Subscribe to the YouTube channel for system design deep-dives:
www.youtube.com/@sriwworldof...
#Azure #DataEngineering #CloudComputing #AzureDataFactory
@powerbitips.bsky.social561 - The best practice articles for the warehouse showed up the same week the cost went up. Call it a coincidence if you want. The read on why both landed together is less generous than that.
@rezarad.bsky.socialAI Functions, Date Tables, and the Future of Data Warehousing
radacad.activehosted.com/index.php?ac...
#MicrosoftFabric #PowerBI #DataWarehouse #DAX #AIAgents #FabricInsider #DateTable #TimeIntelligence #DataEngineering #BusinessIntelligence #DataAnalytics #Difinity
@gabywald.bsky.socialRecherche d'inspirations CyberPunk pour icônes/logos de JdR, jeux de société et vidéo. #JdR #JeuDeRôle #Jeu #JeuDeSociété #JeuVidéo #Graphisme #DataEngineering #OpenToWork #CyberSécurité #CyberPunk #DataEngineering #DataEngineer ... www.linkedin.com/posts/gabrie...
@datahot.xiahongbin.comSnowflake App Runtime 上的 Node.js AI 文档智能
展示了如何把 Snowflake App Runtime、Marketplace 零拷贝数据、ontology 语义层和 Cortex AI 串成可复用的文档智能/问答架构，对在数仓内构建 AI 应用和数据基础有直接参考
DataHot 今日精选 3/5 · 阅读全文：https://datahot.xiahongbin.com/e/880cf15dc545.html?utm_source=bluesky&utm_content=text
#DataEngineering #DataPlatform #数据
@datahot.xiahongbin.comDataHot 今日精选 2/5｜Databricks推出Adaptive Instructed-Retriever，搜索延迟降低2倍
数据从业者关注检索效率与质量平衡，该方案展示了Agent检索层如何用受限多步搜索提升复杂问询能力，对搭建ChatBI/Data Agent有直接参考价值。
阅读全文：https://datahot.xiahongbin.com/e/8eff8be3cc7f.html?utm_source=bluesky&utm_content=card
#DataEngineering #AIAgents #数据
@datahot.xiahongbin.comDataHot 今日精选 1/5｜Databricks 参考模式：嵌入式 AI/BI 仪表盘的逐行安全授权
嵌入式 BI 的核心难题是让不同查看方看到各自数据行；文章给出了基于 Unity Catalog 的统一授权参考模式，可作为多租户嵌入场景的落地依据。
阅读全文：https://datahot.xiahongbin.com/e/ab586bc5d470.html?utm_source=bluesky&utm_content=text
#DataEngineering #BusinessIntelligence #数据
@rajeshdataai.bsky.socialFull Post: www.instagram.com/p/DdEeteCkxx...
.
.
#SQL #interview #DataScience #DataAnalytics #ArtificialIntelligence #data #BusinessAnalyst #DataAnalyst #Database #DataEngineering #Tech #RajeshUmrao
@rajeshdataai.bsky.social
@darryl-ruggles.cloud#AWSLambda #Serverless #DataEngineering #ETL #EventDriven
There are multiple approaches you can take for ingesting data into your AWS account. Many teams start out with an EC2 as it's the simplest. After a while people may notice that the EC2 is idle much of the time and