What is our primary use case?
I provide services related to Data Hub and data management. In total, I can consider my career using Data Hub to be approximately two to two and a half years. I tried to use it on several occasions in previous experiences, and I even started an initiative in my team to have it installed and used broadly in the company because everyone was always lost when it comes to data. They didn't know where the data comes from, which transformation it suffered, and where that data lands. These types of questions were always being asked, and I had the idea to use Data Hub, the open-source version, to give people these analytical capabilities and also integrate with Data Catalog and glossary. I tried to lead that initiative in the past, and then I left the company, so I couldn't see what the end result was.
Right now, Data Hub is already in place, and I use it sometimes, not too frequently because it's maintained by another team, but I use it as an end-user. My company is migrating off the open-source version and is moving to the Atlan version, and I think they use the same tool, but Atlan has more functionalities because they contribute to their private repository.
I used some built-in catalogs on Snowflake, for example, and Databricks, which serve well for that purpose inside their tools. In those cases, you could really see the lineage in Databricks or Snowflake, but at that time, there was no global capability. When we were evaluating Data Hub, there were other companies we were also checking, such as proprietary ones, including Atlan. Although I don't remember all of them, I haven't used those other private companies enough to be confident in my statements about the competitors.
What is most valuable?
I appreciate two things about Data Hub. One thing I appreciate a great deal is the ability that it has to automatically integrate with some of your tools and work as a receptor for receiving data information from your pipeline. For example, if the pipeline starts reading this table and writing these other tables, all this lineage information is sent to Data Hub automatically for some sources, while others only allow a pull-based approach. In those cases, the software needs to push the metrics from time to time to Data Hub, but it allows us to stream all these metrics to Data Hub, which is excellent and is almost seamlessly integrated with some tools that we had.
The second one is the lineage itself, because Data Hub offers a tremendous amount of data lineage capabilities, even column-level lineage, which was very useful for us and is still very useful.
I find Data Hub to be a very important data catalog tool for a company that values data.
What needs improvement?
Regarding what I dislike about Data Hub, I think the UI is minimalistic, and I found myself lost sometimes when looking for a specific dataset. I could find it in the search bar, but multiple things appeared with the same name, making it hard to differentiate if it was a pipeline, a table, or a schema. This UI aspect could use improvement. Additionally, there was another issue regarding configuration; back when I used it, we had to manually input a JSON with all the configurations for a given source, which caused some issues with no visibility on those issues. Mostly the UI is something they could improve.
For how long have I used the solution?
In total, I can consider my career using Data Hub to be approximately two to two and a half years.
What do I think about the stability of the solution?
When I used Data Hub, I did not experience any lagging, crashing, or downtime. I only had some issues with the data refresh that sometimes resulted in outdated information, but other than that, it was good.
What do I think about the scalability of the solution?
Regarding scalability, I think it's good, but you just need to tune it properly to handle the amount of data and data sources you need. When I used it, we didn't have a lot of tables or data entities, and the major issues I experienced were related to collecting metrics. So, while I had some performance issues, they were not related to a lot of tables.
How are customer service and support?
There was no technical support available; we did everything fully on our own.
Which solution did I use previously and why did I switch?
I used the open-source version of Data Hub.
How was the initial setup?
I think the initial deployment was straightforward. When I used it, there were Helm charts already available, so we just had to tweak some minor things, and that was it; it was quite easy.
What about the implementation team?
Data Hub does require some maintenance on my end, such as updates to keep it working. For example, the connectors that are pull-based depend on the volume of the source, and they would fail sometimes. So we had to set up monitoring tools, and whenever they failed, we needed to take a look and maybe tune the pull mechanism to make it more scalable. Other than that, that was the only operational load problem I remember having.
Which other solutions did I evaluate?
We have significant contracts with Google, Amazon, and Microsoft, all of them and our platform, which you can visit at peerspot.com, offers a comprehensive search bar where you can put in almost any product and probably find it, even if there are not a lot of reviews for it. So, Data Hub, as I remember, is a sort of metadata management and AI observability tool.
What other advice do I have?
Currently, I am an end user of Data Hub. I'm not sure about my company's relationship with Data Hub, but personally, I've used it as an enthusiastic user of the Data Catalog subject without any official partnerships. I would rate this product a nine out of ten.
Disclosure: My company does not have a business relationship with this vendor other than being a customer.