Introduction
Data infrastructure is the beating heart of every digital product, and the data it manages comes in all shapes and sizes as do the containers (i.e., databases, data stores) and tools needed to work with it. Database types include OLTP (Online Transaction Processing), OLAP (Online Analytical Processing), In-Memory, Streaming, Time Series, Document, Vector, among others. With many advancements in technology (e.g., programming languages, CPUs), there has been a positive trend in the tech industry recently. This trend aims to replace older alternatives with modern, more efficient, and cost-effective options, ultimately benefiting the end user. Dragonfly is our attempt to innovate the in-memory sector, which had seen only little advancement over the years prior.
However, for fast-growing teams, these data infrastructure projects are often complex to manage, maintain, and operate—sometimes even with dedicated DBAs and infrastructure teams. Databases, being intricate engines, have numerous configurations—and who better to configure and manage them than their creators? Managed offerings effectively remove these challenges from users, transferring them to specialized service providers with expertise—often the original creators of these database technologies—who are best equipped to handle their complexity.
There are several deployment options for data infrastructure today, each with its own set of trade-offs. From managed services provided by major cloud platforms to external database-as-a-service (DBaaS) providers, the key to choosing the right solution lies in understanding these trade-offs. In this article, we will explore the various types of managed offerings available, helping you make informed decisions based on your specific needs.
Types of Managed Offerings
These managed offerings are tailored to the specific database and customer needs. While there's no official classification system, I've categorized these offerings into the following types based on cost structures, usage patterns, and resource allocation.
Dedicated
Here, a specific amount of basic computational resources is dedicated solely to the customer. It's typically a mix of compute, memory, and storage. These resources are physically allocated, always on, and ready to use.
Pricing units are measured in the same terms—for example, X amount of compute with Y of memory costs Z dollars. Since data frequently moves in and out of data infrastructure, network usage is usually charged separately, as it differs from the aforementioned resources. VPC (Virtual Private Cloud) peering is possible, as resources are independently allocated and can be created inside a specific VPC just for the customer and recommended for enhanced security and reduced network costs. This fixed model benefits the user, as there are no sudden shock bills—usage is limited and charged based on provisioned resources.
The desired cloud provider and region are typically configurable. It's advisable to choose the same cloud provider and region as your backend applications to minimize latency and reduce network costs. With no shared infrastructure, all regions for all clouds should be supportable without ROI (return on investment) calculations from a service provider's perspective. While there's no infinite scale, auto-scaling is possible within certain resource limits.
Currently, Dragonfly Cloud operates on this dedicated model.
Multi-Tenancy
Basic resources like compute, memory, and storage are shared across tenants. This means users are allocated higher-level resources such as I/O reads and metric retention's based on the specific product.
Pricing units are not fixed and are based on higher-level primitives derived from resource usage. Network usage is also incorporated into these higher-level pricing units. VPC peering is not possible due to the shared nature of resources.
Cloud and region availability are limited (if offered at all) because multi-tenant infrastructure requires setting up common resources, which may not deliver sufficient ROI for service providers without adequate user demand. Theoretically, infinite scale is possible as these multi-tenant resources can be scaled dynamically. Security and reliability are important aspects of the managed offering provider to consider, as resources are shared.
BYOC (Bring Your Own Cloud)
In this model, the data-plane resources (i.e., what the application or user interacts with) are deployed within the user's account, while the management (i.e., control-plane) is handled by the managed offering provider. This emerging approach has garnered interest from enterprises.
The pricing structure, operational effort, and shared responsibility become more complex, as the user is already directly paying for the data-plane resources. Security, reliability, and latency are significant advantages in this model, as all communication occurs within the customer's account and VPC.
Serverless
In the world of managed database offerings, "serverless" is challenging to categorize. While it's essentially a form of multi-tenancy, the underlying infrastructure can be ephemeral, and the pricing model is based on pay-per-usage, making it efficient in terms of resource allocation and cost reduction. From the user's perspective, the database is inherently single-tenant, offering a single, elastically scaling database to work with.
In this model, multi-tenancy is shifted to a different level—usually at the node. Multiple customer databases are packed onto the same node but load-balanced based on usage at the process level.
While the serverless model promises cheaper pricing through dynamic load balancing and efficient resource usage, the pursuit of infinite scalability and fully automated elasticity—features that organizations with a steadily and predictably growing user base often don't need—can come at a significantly higher premium. Learn more about serverless pricing in our previous blog post. In the meantime, for databases, this approach is more complex to implement and often comes at the cost of reliability and potential "noisy neighbor" issues.
Dedicated - Dragonfly's First Managed Offering
When designing our first managed offering at Dragonfly, we chose the dedicated model for several key reasons that align with our philosophy of providing transparent, efficient, and reliable solutions to our users.
- Cost-Effectiveness: One of our priorities is offering clear, easy-to-understand, and affordable pricing. With the dedicated model, we charge based on provisioned memory, providing transparency without unexpected costs. Users know exactly what they are paying for, and it's a cost-efficient option.
- Use Case Suitability: A single Dragonfly instance is incredibly powerful, often enough to handle the needs of most applications. Our design makes Dragonfly particularly suitable for caching use cases—which is one of the major use cases—with automatic eviction and upcoming features like SSD data tiering. If memory limits are reached, the eviction process ensures that performance remains optimal, meaning automatic scaling is not immediately necessary in many cases.
- Speed and Security: We built Dragonfly to minimize latency. By placing Dragonfly instances right next to your backend applications within the same cloud region, we eliminate network latency, which is often the largest contributor to delays in data infrastructure. Additionally, the use of VPC peering ensures that traffic stays within your cloud environment, providing top-tier security while maintaining exceptional speed.
- Simplicity and Reliability: The dedicated model is easier to manage and maintain, as each customer has their own isolated resources, avoiding potential "noisy neighbor" issues common in multi-tenant environments. This model offers a straightforward setup that aligns with Dragonfly's emphasis on performance and reliability.
This combination of cost transparency, powerful performance, minimal latency, and enhanced security makes the dedicated model a natural first choice for Dragonfly. It ensures that our users receive the best balance between efficiency, cost, and performance, tailored to meet the needs of fast-growing teams and applications with predictable workloads.
Conclusion
Selecting the right managed offering depends on your organization's specific requirements and growth trajectory. For teams seeking predictable costs and enhanced security, the dedicated model is often a reliable and straightforward choice, offering convenience for most customers. If automatic scalability and elasticity are top priorities, multi-tenancy and serverless options offer flexibility but may come with trade-offs in transparency and pricing. The BYOC approach provides a blend of control and security, ideal for enterprises that want to manage data in their own environments, though it may introduce added complexity.
Ultimately, the right decision balances infrastructure complexity with operational needs, ensuring scalability and cost-effectiveness without overburdening your teams.
