Dremio Reviews

I have used this solution as an ETL tool to create data marks on data lakes for bridging. I have used it as a greater layer for ad-hoc queries and for some services which do not require sub-second latency to credit data from very big data lakes. I have also used it to manage simple ad-hoc queries similar to Athena, Presto or BigQuery. 

We do not have a large number of people using this solution because it's mainly setup as a service to service integration. We integrated a big workload when we started using Dremio and this was very expensive. The migration is still in progress. As soon as this migration is finished, we plan to migrate ad-hoc queries from our analytical team. 

The native error interface has been most valuable. With Apache Airflow, which Dremio supports, you don't need to digitalize data. Dremio gives you the ability to create services which do not require additional resources and serialization. 

Dremio doesn't support the Delta connector. Dremio writes the IT support for Delta, but the support isn't great. There is definitely room for improvement. 

The community version lacks information or documentation. If you want to create a more dynamic scaling policy for your cluster, it's always a problem because you need an additional custom code to be sure that Dremio shuts down gracefully and does not get interrupted.

The support of the Delta I/O connector could be improved because the current support is very basic and the performance isn't great. 

I have used this solution for three years in a non-production environment and for nine months in a production environment. 

This is a stable solution.

I have used the community forum to get most of my answers for Dremio. This is best suited for simple queries but does not work as well for advanced questions that need input from a developer. 

The complexity of the setup depends on the use case. If you are working with a cluster that you plan to shut down after a week, the setup is simple. For more complicated scenarios where you need to connect parquet table, it is more complicated and the documentation to support this setup is not comprehensive. 

Experienced DevOps teams need to spend at least three to five days to understand the level of ability between multiple masters and how the scheduling works. 

I tried to integrate my Dremio metrics and logs to DataDog and it wasn't simple.

Right now the cluster costs approximately $200,000 per month and is based on the volume of data we have.

We considered Apache Drill, Apache Presto, Flink and Spark. The main reason we chose Dremio was cost and multitenancy.

I would advise others not try to use Dremio as an ETL framework from day one. I also recommend do not use reflections, or any other tool which make dremio cluster statefull.

The installation for a production environment is complex and I would not recommend this solution for small businesses.

I would rate this solution a seven out of ten. 

We were using Dremio as a data lake query engine tool. We were creating our PDSs and VDSs on top of our S3 buckets, and our data lake and the data-scientist teams were using the data for further processing. We didn't use it for any ETL jobs. We were using it as a data-lake tool.

The most valuable feature of Dremio is it can sit on top of any other data storage, such as Amazon S3, Azure Data Factory, SGFS, or Hive. The memory competition is good. If you are running any kind of materialized view, you'd be running in memory.

Dremio takes a long time to execute large queries or the executing of correlated queries or nested queries. Additionally, the solution could improve if we could read data from the streaming pipelines or if it allowed us to create the ETL pipeline directly on top of it, similar to Snowflake.

I have been using Dremio for approximately two years.

Dremio is highly stable.

Dremio is scalable which is a benefit of the solution. You can scale up to the number of instances you want in case you are feeling the load, and in case you feel your query is running low or you are receiving extra traffic.  

You can set the configuration while installing and while setting it up in the Dremio. In the configuration file, you can set up a lot of settings, such as what time.

We have approximately 18 people using the solution in my organization.

We have been in touch with the support from Dremio when we had some internal issues. This happened approximately two times. The support is good.

This is the first tool in this category that I have used.

Dremio's initial setup took one or two days, one day is sufficient and typical.

There was one DevOps person used for the deployment and maintenance of the solution.

Dremio is less costly competitively to Snowflake or any other tool.

My advice to others is if they are creating a data lake for a customer, Dremio would be useful for a data engineering team. If they're willing to create a data lake and they wanted to use it with the cloud-agnostic tool, then it is a good choice. If this solution meets their requirement they should try it out.

I rate Dremio an eight out of ten.