Technology Prophecy's generative AI assistant ushers in a new era...

Prophecy’s generative AI assistant ushers in a new era of data pipeline automation


- Advertisment -

Join top executives in San Francisco on July 11-12 to hear how leaders are integrating and optimizing AI investments for success. Learn more

Data engineering startup Prophecy gives a new twist to creating data pipelines.

The California-based company, known for its low-code SQL tooling, today announced data copilot, a generative AI assistant that can create trusted data pipelines from natural language prompts and improve pipeline quality with greater test coverage.

The capability has the potential to make pipeline development a breeze and save data engineers valuable time for other more pressing tasks. Previously, we’ve seen gen AI used for other aspects of data workflows, such as querying and cataloging data.

Along with the tool, Prophecy announced a new platform to help companies build gen AI applications on top of their private data.


Transform 2023

Join us on July 11-12 in San Francisco, where top executives will talk about how they integrated and optimized AI investments for success and how they avoided common pitfalls.

register now

How does data copilot help?

Historically, building a data pipeline has revolved around writing complex SQL code. Data engineers had to put in a lot of time and effort to bring business users’ desired pipelines to life. Then players like Prophecy came in, offering low-code solutions (a drag-and-drop visual canvas) to simplify things.

Now, as the next step in this work, Prophecy has introduced the data copilot, which only requires someone to say what they want in natural language.

Once a command is given, the platform uses it to suggest a pipeline that brings all the data together for the desired report. The user can then preview the pipeline and accept it, or decline to come up with anything else.

“This will enable companies to be less of a data engineering bottleneck as business data users and others can serve themselves… Further, these data products will also be made more consistent and of higher quality as Prophecy Data Copilot suggests transformations and expressions. too,” Raj Bains, CEO and co-founder of Prophecy told VentureBeat.

To accomplish this, Bains explained that Prophecy creates a comprehensive knowledge graph of a company’s data models that includes technical metadata associated with tables and schemas, business metadata from data catalogs, and historical queries and code run on SQL, Spark, or Airflow. This graph is then fed into an advanced large language model that translates the natural language user query into a performant data pipeline. The system also learns and improves based on user feedback, he added.

Platform to build gen AI apps

In addition to the data copilot, Prophecy adds a platform to build gen AI solutions like chatbots on top of private company data.

The offer works in a two-step process. First, a data engineer populates a knowledge warehouse with unstructured text in internal messaging systems, documents, support tickets, and more (converted to vectors). Next, the engineer builds a streaming inference pipeline, i.e. a chatbot, supported by OpenAI.

“When a user asks a question, a vector lookup is performed to retrieve the internal documents most relevant to that question,” Bains explains. “Those documents provide crucial context for answering the question posed. This context plus the question itself form the prompt, which is sent to OpenAI via APIs. The response is then sent back to the user.”

Both new offerings are available to use starting today.

VentureBeat’s mission is to become a digital city plaza where tech decision makers can learn about transformative business technology and execute transactions. Discover our Briefings.

Shreya Christina
Shreya has been with for 3 years, writing copy for client websites, blog posts, EDMs and other mediums to engage readers and encourage action. By collaborating with clients, our SEO manager and the wider team, Shreya seeks to understand an audience before creating memorable, persuasive copy.


Please enter your comment!
Please enter your name here

Latest news

Vulkan Vegas 50 free spins 50 spinów na Book of Dead!

Wystarczy zarejestrować się przez nasz link, aby odebrać ekskluzywny bonus powitalny, czyli 50 darmowych obrotów. Później pozostaje już je...

Vulkan Vegas bonus za rejestrację Odbierz darmowy bonus

Druga część bonusu to 25 darmowych spinów, które można wykorzystać w automacie Book of Dead. Oprócz bonusu powitalnego, Vulkan...

Odkryj Emocje Związane z Grami Online

Wszystkie Twoje wygrane są utrzymywane w tajemnicy i nikt inny nie zna Twojego adresu IP ani kwoty, którą wygrałeś...

Sprawdź Najlepsze Kasyno Online w Polsce

Przy pierwszym uruchomieniu strony użytkownik otrzymuje informacje o niezwykle przystępnej ofercie bonusowej. Do tego dochodzi ogromna, bardzo dynamicznie rozwijająca...
- Advertisement -

Online Dating in the Modern Era

The world of online dating has revolutionized the way people connect and find love. With the advent of technology,...

Mostbet dən pis bukmeker kontorunun əlamətləri Təhsil Portalı

Contentin Virtual İdman mərcləriMostBet bukmeyker şirkəti ölkəmzidə niyə bloklanmır?"Əş-Şəbab"ın liderlərindən biri Somalidə hücum nəticəsində yaralanıbMostBet-də oyunçular üçün təqdim olunan...

Must read

- Advertisement -

You might also likeRELATED
Recommended to you