Elasticsearch

Elasticsearch is a source-available search engine developed by Elastic. It is based on Apache Lucene and provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Official clients are available in Java, C#, PHP, Python, Ruby, and other languages. According to the DB-Engines ranking, Elasticsearch is the most popular enterprise search engine.[2]

Elasticsearch is distributed and uses JSON documents stored in indices divided into shards, each of which may have replicas distributed across cluster nodes. It supports full-text search, faceted search, real-time search, and multitenancy. The software is developed alongside Logstash, Kibana, and Beats as part of the Elastic Stack (formerly the ELK Stack).

History

Shay Banon created the precursor to Elasticsearch, called Compass, in 2004.[3] Developing a third version of Compass, he concluded that a full rewrite was necessary to build a scalable, distributed search solution using JSON over HTTP as a common interface.[3] He released the first version of Elasticsearch in February 2010.[4]

Elastic NV was founded in 2012 to provide commercial services and products around Elasticsearch and related software. In June 2014, the company raised $70 million in a Series C funding round led by New Enterprise Associates, with additional funding from Benchmark Capital and Index Ventures, bringing total funding to $104 million.[5]

In March 2015, the company changed its name from Elasticsearch to Elastic and, through the acquisition of Found, launched a managed cloud offering later known as Elastic Cloud.[6] In November 2017, Elastic acquired the search startup Swiftype, whose technology became the basis for Elastic App Search and Elastic Site Search.[7] Elastic also formed partnerships with Google to offer Elastic Cloud on Google Cloud Platform and with Alibaba to offer Elasticsearch and Kibana on Alibaba Cloud.

In June 2018, Elastic filed for an initial public offering with an estimated valuation of between $1.5 and $3 billion.[8] On 5 October 2018, Elastic was listed on the New York Stock Exchange.[9]

Licensing changes

In January 2021, Elastic announced that starting with version 7.11, Elasticsearch and Kibana would be relicensed from Apache License 2.0 to a dual license under the Server Side Public License and the Elastic License, neither of which is recognised as an open-source license.[10][11] Elastic stated the change was a response to Amazon Web Services (AWS) offering Elasticsearch and Kibana as a service without what Elastic described as adequate collaboration.[11][12] Critics of the change noted that Elastic had previously promised never to alter the Apache 2.0 licensing of Elasticsearch, Kibana, Beats, and Logstash, and predicted the move would damage the ecosystem.[11]

AWS responded by announcing a fork of the projects to continue development under Apache License 2.0.[13] Other members of the Elasticsearch ecosystem, including Logz.io, CrateDB, and Aiven, also committed to supporting a fork.[14][15][16] Due to potential trademark conflicts with the name "Elasticsearch", AWS rebranded their fork as OpenSearch in April 2021.[17]

In August 2024, Elastic added the GNU Affero General Public License as a third licensing option for Elasticsearch starting with version 8.16.0.[18][19] This made the software available under a free and open-source license once again alongside the two source-available options.

Architecture and features

Elasticsearch is built on Apache Lucene and exposes Lucene's capabilities through a JSON and Java API. Documents are stored in indices, which are divided into primary shards; each shard may have zero or more replicas distributed across cluster nodes. Routing and rebalancing are handled automatically. Once an index has been created, the number of primary shards cannot be changed.[20]

The engine supports faceted search and percolation, a form of prospective search in which stored queries are matched against incoming documents rather than the reverse.[21] A gateway module handles long-term index persistence, allowing an index to be recovered after a node failure.[22] Real-time GET requests make Elasticsearch usable as a NoSQL datastore, though it does not support distributed transactions.[23]

In May 2019, Elastic made the core security features of the Elastic Stack available without charge, including TLS for encrypted communications, file and native realm authentication, and role-based access control for cluster APIs and indices.[24] The corresponding source code is available under the Elastic License.[25] Elasticsearch also offers SIEM[26] and machine learning capabilities[27] as part of its commercial offerings.

Official client libraries are maintained for Java,[28] C# (.NET),[29] PHP,[30] Python,[31] and Ruby,[32] among others.

Elastic Stack

Elasticsearch is developed alongside Logstash (a data collection and log-parsing engine), Kibana (an analytics and visualisation platform), and Beats (lightweight data shippers). The four products are designed for use together as the Elastic Stack.[33] The combination was previously known as the ELK Stack, an initialism for "Elasticsearch, Logstash, and Kibana".

See also

  • Information extraction
  • Kibana
  • Logstash
  • Grafana
  • List of information retrieval libraries
  • OpenSearch (software) - open-source fork of Elasticsearch maintained by AWS

References

  1. Releases · elastic/elasticsearch retrieved 16 November 2025^
  2. DB-Engines Ranking - popularity ranking of search engines db-engines.com, retrieved 10 January 2016^
  3. Shay Banon. The Future of Compass & ElasticSearch 7 July 2010^
  4. Shay Banon. You Know, for Search 8 February 2010^
  5. Ron Miller. ElasticSearch Scores $70M In Series C To Fund Growth Spurt TechCrunch, 5 June 2014, retrieved 4 April 2024^
  6. Andrew C. Oliver. Elasticsearch buys into search as a service, rebrands as 'Elastic' InfoWorld, 10 March 2015, retrieved 1 April 2019^
  7. Anthony Ha. Elastic acquires search startup Swiftype TechCrunch, 9 November 2017, retrieved 3 April 2024^
  8. Theodore Schleifer. The IPOs keep coming: The search company Elastic has filed to go public Recode, 21 June 2018, retrieved 22 June 2018^
  9. Shay Banon. Ze Bell Has Rung: Thank You Users, Customers, and Partners Elastic NV, 5 October 2018, retrieved 24 October 2018^
  10. Shay Banon. Doubling down on open, Part II Elastic, 14 January 2021, retrieved 19 January 2021^
  11. Steven J. Vaughan-Nichols. Elastic changes open-source license to monetize cloud-service use ZDNet, retrieved 23 January 2021^
  12. Shay Banon. Amazon: NOT OK - why we had to change Elastic licensing Elastic, 19 January 2021, retrieved 19 January 2021^
  13. Stepping up for a truly open source Elasticsearch Amazon Web Services, 21 January 2021, retrieved 28 January 2021^
  14. Steven J. Vaughan-Nichols. AWS, as predicted, is forking Elasticsearch ZDNet, retrieved 28 January 2021^
  15. CrateDB Doubling Down on Permissive Licensing and the Elasticsearch Lockdown CrateDB, 27 January 2021, retrieved 28 January 2021^
  16. Momentum Builds to Break Elasticsearch Licensing Deadlock Datanami, 25 January 2021, retrieved 31 January 2021^
  17. Tim Anderson. You know what? Fork this: AWS renames its take on Elasticsearch to OpenSearch following trademark fight The Register, 13 April 2021, retrieved 13 April 2021^
  18. Shay Banon. Elasticsearch is Open Source, Again 29 August 2024^
  19. Add AGPLv3 as a supported license · elastic/elasticsearch@0279c0a GitHub, retrieved 13 April 2025^
  20. How to monitor Elasticsearch performance Datadog, 26 September 2016, retrieved 26 September 2016^
  21. Percolate API Elasticsearch.org, retrieved 4 February 2014^
  22. elasticsearch Guide: Gateway Elasticsearch.org, retrieved 19 April 2013^
  23. No transaction support Elasticsearch-users.115913.n3.nabble.com, 8 July 2010, retrieved 4 February 2014^
  24. Security for Elasticsearch is now free Elastic Blog, 20 May 2019, retrieved 17 June 2019^
  25. Doubling Down on Open Elastic Blog, 27 February 2018, retrieved 24 October 2019^
  26. Introducing Elastic SIEM Elastic Blog, 25 June 2019, retrieved 2 March 2020^
  27. Introducing Machine Learning for the Elastic Stack Elastic Blog, 4 May 2017, retrieved 2 March 2020^
  28. Elasticsearch Java Client GitHub, retrieved 7 October 2022^
  29. Elasticsearch .NET Client GitHub, retrieved 7 October 2022^
  30. Elasticsearch PHP Client GitHub, retrieved 7 October 2022^
  31. Elasticsearch Python Client GitHub, retrieved 7 October 2022^
  32. Elasticsearch Ruby Client GitHub, retrieved 7 October 2022^
  33. Ron Miller. Elastic brings order to its product line with Elastic Stack TechCrunch, 26 October 2016, retrieved 3 April 2024^