Menu
LucidWorks preps Solr stack as Splunk killer

LucidWorks preps Solr stack as Splunk killer

LucidWorks combines the LogStash log engine with the highly scalable Lucene search engine

Enterprise search software provider LucidWorks has introduced a package based on its open-source Apache Solr search engine that will allow administrators and business analysts to extract more information from IT system logs.

SiLK "is a solution that relies on open core components that organizations can use to manage log data at scale," said Will Hayes, LucidWorks chief product officer.

The SiLK package combines Apache Lucene/Solr with a number of open-source analysis tools, namely Apache Flume, LogStash and Kibana.

Kibana offers the reporting visualization capabilities and LogStash is used to collect, store and parse logs. Flume provides a way to connect with Hadoop repositories. Apache Solr, which LucidWorks oversees, provides the searching and indexing capabilities. Solar used to be called Lucene/Solr, after two technologies that were combined, before the name was shortened to Lucene. LucidWorks employs about 25 percent of the core developers who manage and update Lucene and Solr.

The software package could help in security analysis, business intelligence, fraud detection and other use cases, according to the company. It can offer time-series analysis, data discovery and correlation.

The open source software stack that powers SiLK is not new -- many organizations have already combined LogStash and Kibana to analyze log data. Most implementations have used another open source search engine, Elasticsearch, however.

"A lot of people out in the community were talking about using LogStash with Solr," Hayes said. "A number of organizations are running into issues with using Elasticsearch at scale."

LucidWorks says that using Lucene instead of Elasticsearch will allow an organization to aggregate and search across more data, Hayes said. SiLK is aimed at organizations that have anywhere from hundreds of gigabytes to terabytes of data to ingest daily.

According to Hayes, SiLK can also provide some scalability advantages over the commercial market leader of log-data analysis, Splunk. Splunk charges, at least in part, based on how much data is being analyzed, which can add up when dealing with extremely large amounts of data.

SiLK also works well with Hadoop deployments, Hayes said. It has been certified to work with the Cloudera Enterprise 5 commercial Hadoop package and Solr is frequently incorporated into other Hadoop distributions, such as those offered by MapR and Hortonworks.

SiLK will be a component of the Lucene subscription services that LucidWorks offers, which start at about US$6,000 per node.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is Joab_Jackson@idg.com


Follow Us

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags Data managementsoftwareapplicationsdiagnosticssystem managementdata miningUtilitiesLucidWorks

Featured

Slideshows

Meet the Reseller News 30 Under 30 Tech Awards 2020 winners

Meet the Reseller News 30 Under 30 Tech Awards 2020 winners

This year’s Reseller News 30 Under 30 Tech Awards were held as an integral part of the first entirely virtual Emerging Leaders​ forum, an annual event dedicated to identifying, educating and showcasing the New Zealand technology market’s rising stars. The 30 Under 30 Tech Awards 2020 recognised the outstanding achievements and business excellence of 30 talented individuals​, across both young leaders and those just starting out. In this slideshow, Reseller News honours this year's winners and captures their thoughts about how their ideas of leadership have changed over time.​

Meet the Reseller News 30 Under 30 Tech Awards 2020 winners
Reseller News Exchange Auckland: Beyond the myths — how partners can master cloud security

Reseller News Exchange Auckland: Beyond the myths — how partners can master cloud security

This exclusive Reseller News Exchange event in Auckland explored the challenges facing the partner community on the cloud security frontier, as well as market trends, customer priorities and how the channel can capitalise on the opportunities available. In association with Arrow, Bitdefender, Exclusive Networks, Fortinet and Palo Alto Networks. Photos by Gino Demeer.

Reseller News Exchange Auckland: Beyond the myths — how partners can master cloud security
Reseller News welcomes industry figures at 2020 Hall of Fame lunch

Reseller News welcomes industry figures at 2020 Hall of Fame lunch

Reseller News welcomed 2019 inductees - Leanne Buer, Ross Jenkins and Terry Dunn - to the fourth running of the Reseller News Hall of Fame lunch, held at the French Cafe in Auckland. The inductees discussed the changing face of the IT channel ecosystem in New Zealand and what it means to be a Reseller News Hall of Fame inductee. Photos by Gino Demeer.

Reseller News welcomes industry figures at 2020 Hall of Fame lunch
Show Comments