Cloud file companies startup LucidLink tripled in measurement over the previous 12 months, preserving remote media content material workers fed fast with information via sub-file motion and caching, and it has zero egress prices with IBM COS and a low-cost cloud storage tier coming. We have been briefed by CEO Peter Thompson and take a complete have a look at his firm and its know-how in this two-part article.
Peter Thompson
LucidLink’s Filespaces product provides remote workers’ laptop computer and desktop programs with file information sourced from an underlying cloud object retailer distributed in part-file chunks to hurry information entry. It synchronizes the file system state throughout the working groups’ programs utilizing native metadata shops. Thompson stated that the pandemic offered a lift to its gross sales and modified its improvement path to match the brand new working-from-home type of many media inventive and enhancing folks.
Thompson informed B&F: “On pretty much every metric that we track over the last year even during this challenging recession, and contractions going on and revaluations and everything else, we’ve managed to triple the business.”
LucidLink is a 2016 startup working in the identical normal area as CTERA, Nasuni and Panzura however has been flying a bit below the radar.
Co-founders Thompson and CTO/chairman George Dochev found that “collaborating on high velocity files through fast rates of change, especially when those files became big, became just harder and harder and harder. It was an unsolved problem.”
They tried “to figure out what is the problem, because there’s lots of bandwidth, latencies are reasonable, there was there’s just kind of no technical reason that we could see that made this such a painful process when you try to access files over distance.”
“So we threw everything we knew out the window, and began looking at how can you access files over a long, thin connection. And that’s really what we’ve done: we’ve made a kind of a cloud-optimized file system, that promotes that type of access.”
The first use case they noticed was “pattern matching for distributed teams who collaborated on large sets of data, and specifically on big files. And this crops up in a lot of different areas: in video creation and creative content; in the AEC, space for architectural design and CAD CAM files; in medical imaging; in life sciences research; in geospatial models in the oil and gas industry. In all kinds of places this pops out.”
It’s additionally energetic in sports activities broadcasting and company advertising and marketing departments for video manufacturing. That is its quickest rising enterprise section.
Product and pandemic
The primary product was constructed by 2020 and LucidLink was beta testing it in these vertical markets when COVID-19 struck, unexpectedly ramping up curiosity in its know-how. Thompson stated: “It was at the beginning stages of the pandemic when suddenly, we began receiving inquiries from media teams from broadcast people.”
Thompson stated LucidLink was informed: “We sent our editors home to work from there, gave them a laptop, a VPN connection and a hard drive. And that’s sort of getting us through the next couple of weeks, but then we don’t know what we’re gonna do.”
These clients “would try LucidLink’s Filespaces, usually on a demo, and it would just roll directly into production.”
This simply snowballed. “We began having large customers – you know, kind of in that $100,000 range customers – who began immediately using us in production for distributed teams across the country, or even across the globe. And we grew our business from 2020. Boy, it grew so fast. We were so small at that time, you know, it was maybe a 10x, or 20x, or something kind of growth to 2021.”
This validated LucidLink’s strategy of concentrating on massive recordsdata. Thompson stated: “We knew that the larger the file was, the more distributed the team, the quicker they would see the magic.”
If recordsdata have been small and datasets not that large, as with sharing workplace recordsdata and issues like PDFs, then file sync and share was developed for that again in the day and is ok. Prospects aren’t going to note that a lot of a distinction from it with LucidLink’s FileSpaces.
FileSpaces comes into its personal with bigger recordsdata and datasets. “LucidLink is giving you the entire set of data. It’s allowing you to mount your cloud drive as if it’s local, and the application and the user don’t have any idea where the files are actually located.”
What Lucid’s tech isn’t doing is having a consumer “click on the file, download the entire thing, edit it and push it all back up to the cloud, and then synchronize that across to everywhere else.” As a substitute “you’re actually using it in place. Or, more accurately, we’re distributing the file, we’re downloading portions of it, you’re using your active set.”
“Through the way that we’ve done this distributed file system, the source of truth remains in the cloud, but you’re able to use the file as if it’s local. And more importantly, you’re able to collaborate with other users who are distributed around in different areas.”
No workflow adjustments
What this meant for the media editors was that there was no change. “I think our success was driven by those editors and the users. They really embraced this because they didn’t have to change any of their workflow. They didn’t have to learn new tools, they had the muscle memory that was intact, and in place. They used to be in the office, connected to the LAN, going to the NAS or the file server. Now they’re at home, connected to the internet, going to the Lucid mount. And there was no difference.”
“The performance was so great that they just forgot about it. We would just fade into the background, which is exactly what we wanted to do. We wanted to disappear. The more we could disappear, and have them back in their workflow and on their tools, without even thinking about us, the more we knew we’d be successful.”
Lucid modified its hiring sample. “We began hiring less storage and networking specialists, and more workflow specialists – people who knew that knew the tool sets and the workflow for for creative content, and edit and video editing.”
This match the work-from-home scene and the big recordsdata and datasets market meant LucidLink stored on rising. “It hasn’t shown any signs of showing slowing down. In fact, the velocity is still increasing. I think the reason for this is that it has shown to be such a high productivity increase for these teams. To put it quite simply, they’re no longer waiting for files to download.”
Metadata streaming
Nonetheless, the elevated measurement of the enterprise brought about a know-how rethink. “We realized that we were going to hit some scalability issues at some point. And we knew that we needed to be able to support a billion files and tens of thousands of users. The initial product wasn’t quite going to do that, at least not comfortably.”
“In late 2021, through to mid-2022, we reworked our platform. This is what we we termed Lucid 2.0. And the intent of this was to increase performance, but more importantly, to allow for this kind of massive scale in numbers of files that we could support and numbers of users that could be in the system.”
This concerned introducing metadata streaming. “Because these file systems were getting so large, the metadata associated with them was becoming a big data problem in and of itself. And then we said, alright, we can’t continue going on synchronizing metadata across tens of thousands of clients – that’s not efficient.”
“So we did the same type of streaming technology that we did with the file data for the metadata. That’s what we call our metadata streaming. This means that, even if someone joins fresh on a very large file space, they’ll have immediate access to it.”
After the LucidInk 2.0 platform change “we began focusing our efforts on specific user requests. And we began rolling out some features,” Thompson stated.
One was single sign-on (SSO) and a second was a a remote add indicator to say when a file is is totally loaded and able to go. One other one is full path shares: “The ability to show a path to your assets, within your tools consistently across different sets of tools, and different platforms, and different operating systems.”
A fourth is a public API in order that Filespaces can combine with infrastructure instruments equivalent to Adobe Premiere or Autodesk Flame. If clients must go to a drive, a mount or a NAS, they will go to Filespaces as a substitute, which exhibits up as a drive. Thompson stated: “We can be that centralized, consistent repository that spans across all of those tools, and therefore that entire workflow. The amount of pushing data from one point into another during that life cycle is dramatically reduced.”
The mixing with media asset administration programs can be improved due to this API. Different roadmap gadgets embrace an internet shopper and a cell machine expertise.
***
Half two of this account of LucidLink and its know-how will have a look at IBM COS, egress fees, direct object entry and LucidLink’s competitors.
0 Comments