Skip to main content

Introduction

What is VulcanSQL​

VulcanSQL empowers you to build robust and scalable APIs by leveraging the power of SQL templates, similar to how you use SQL templates in dbt to transform data. With VulcanSQL, you can encapsulate your logic and transformations within SQL templates, enabling you to effortlessly create advanced Data APIs.

vulcan overview

Whether you need to filter data from a data warehouse, perform data validation on API parameters, or handle errors, VulcanSQL offers the flexibility and simplicity to accomplish your objectives effectively. By utilizing SQL templates, you can streamline your API development process, ensure the accuracy and integrity of your data, and deliver powerful analytics capabilities to your users.

Why VulcanSQL?​

Given the vast amount of analytical data in databases, data warehouses, and data lakes, there is currently no easy method for data professionals to share data with relevant stakeholders for operational business use cases.

Without a specialized framework to streamline the creation of APIs for AI agents and apps to interact with databases and data warehouses, developers today must undertake a more manual and complex process. This approach comes with several pain points and challenges:

Traditional Approach and Pain Points​

1. Custom API Development:​

  • Time-Consuming: Developers need to manually code the APIs, which can be time-consuming, especially for complex applications or when dealing with multiple data sources.
  • Error-Prone: Manual coding increases the risk of bugs and errors, which can affect the reliability and performance of the APIs.

2. Integration Complexity:​

  • Diverse Data Sources: Integrating multiple data sources with different formats and protocols requires significant effort and expertise.
  • Lack of Standardization: Without a standardized approach, each API might follow different conventions, making it harder for AI agents to interact with them consistently.

3. Security and Compliance:​

  • Security Risks: Ensuring that APIs are secure and that data access complies with regulations (e.g., GDPR, HIPAA) requires additional layers of work, including authentication, authorization, and data encryption.
  • Maintenance Overhead: Security and compliance requirements can evolve, necessitating ongoing maintenance and updates to the APIs.

4. Scalability and Performance:​

  • Scalability Concerns: Custom-built APIs may not be optimized for scalability, leading to performance issues as the number of requests or the volume of data grows.
  • Resource Intensive: Optimizing for performance and scalability can require significant resources, both in terms of development time and infrastructure.

5. Documentation and Usability:​

  • Lack of Documentation: Properly documenting APIs for easy understanding and use by other developers or AI agents can be overlooked or undervalued.
  • Usability Issues: Without clear and comprehensive documentation, it becomes challenging for others to integrate with and effectively use the APIs.

With VulcanSQL you can​

  • Rapid Development and Integration: By abstracting the complexities of directly interacting with databases and data warehouses, developers can focus on the higher-level logic of their applications. This reduces the development time and simplifies the process of integrating AI capabilities into applications.

  • Standardization: Utilizing OpenAPI documents for interaction provides a standardized way for AI agents to understand and interact with different APIs. This promotes interoperability among various systems and tools, making it easier to integrate with a wide array of services and data sources.

  • Scalability and Maintenance: A template-driven approach to API creation can make it easier to scale and maintain APIs over time. Changes in the underlying data schema or business logic can be propagated to the APIs more efficiently, without the need for extensive manual adjustments.

  • Accessibility: Making data more accessible to AI agents through well-defined APIs can unlock new insights and capabilities by leveraging machine learning and analytics. This can enhance decision-making processes and automate routine tasks, among other benefits.

Main Features​

To fully utilize the capabilities of VulcanSQL, it is essential to familiarize yourself with the available features and functionalities. The documentation is organized into several sections, each covering an important aspect of the framework. Here's a brief overview of the main features:

Building Data API​

  1. Connecting to Data Sources: Discover how to seamlessly integrate VulcanSQL with your data warehouse to unlock the power of analytics.

  2. Writing SQL Templates: Leverage the power of SQL templates to encapsulate your logic and transformations. Discover how to structure and write SQL templates that serve as the foundation for your data APIs, enabling you to manipulate and transform data with ease.

  3. Caching Datasets: Explore techniques for caching datasets in VulcanSQL to optimize query performance and reduce the load on data sources. This feature enables you to store and reuse intermediate results, improving overall API response times.

  4. Handling Errors: Learn how to handle errors effectively in VulcanSQL. This section guides you through error handling strategies, including throwing errors when parameters are invalid or when data is missing.

  5. Validating API Parameters: Discover how to validate and sanitize API parameters in VulcanSQL. This feature ensures that the input data is validated and meets the required criteria, reducing the risk of incorrect or malicious requests.

  6. Handling Data Privacy: Explore the mechanisms and practices for handling data privacy in VulcanSQL. It encompasses practices and measures implemented to safeguard personal, confidential, or regulated information from unauthorized access, misuse, or disclosure.

Extensions​

VulcanSQL allows you to extend its core functionalities through extensions.

  1. dbt: VulcanSQL supports queries from dbt's SQL models directly. If you use dbt to create some models, you can make APIs for them instantly.
  2. Hugging Face: With this plugin, you can leverage the power of language models to generate SQL quries using natural language.
  3. API: You can access data from 3rd parties through calling REST APIs with this extension.

API Catalog & Documentation​

  1. API Catalog: Learn how to create an API catalog with VulcanSQL, providing a centralized repository for all your Data APIs. This catalog enables easy discovery, management, and sharing of APIs within your organization.

  2. API Documentation: Understand how to generate comprehensive API documentation (OpenAPI Specification) with VulcanSQL, including endpoints, parameters, response formats, and examples.

API Configuration​

Discover the extensibility of VulcanSQL through API plugins. This section explains how to leverage pre-built plugins to enhance the functionality of your Data APIs, including Response Format, Pagination, CORS, Rate Limit and Access Log.

Deployment and Maintenance​

Understand the deployment and maintenance considerations for VulcanSQL in a production environment. This section covers topics such as packaging and deploying your APIs.