Getting Started with Apache Solr 7: Part 1

What is Solr?

In simple words, it is a search engine. Similar to a relational database system (like MySQL etc.), it can store textual, numeric, spatial or binary data and allows quick search and retrieval. Here’s a list of the equivalent concepts between a database system and Solr:

Solr is suitable for searching, filtering and faceting across full text fields and other types of fields and flexible for influencing ranked order of retrieved results based on relevance to the queries. A major difference between Solr and database systems is that Solr is not suitable for join operations across multiple collections, unlike database systems where joins across tables are very common. Database systems practitioners suggest data to be normalized, but it is recommended to have your data as de-normalized as possible in Solr. Solr offers tons of other features like searching across multiple fields at once, spell correction, highlighting, grouping, streaming functions, robust scaling features etc. We shall explore all of those in subsequent posts.

Why this guide?

Since Solr 7, there are newer and cleaner ways to interact with Solr. That makes the starting experience a lot easier than it used to be. The new V2 APIs at /api endpoint offer a cleaner way to access Solr. Also, Solr supports JSON by default. Apart from this, collection creation is much simpler as the user no longer needs to upload a configset before creating a collection.

Since the early days of Solr, there have been various ways to achieve the same objectives in Solr across different versions. Therefore, earlier tutorials and guides offer various different styles of accessing the APIs. The intention of this tutorial, and subsequent ones here, is to present an easy, modern and clean way to interact with Solr.

Running Solr 7

For this tutorial, lets use Docker to start up Solr 7.1, create a collection, index a few documents and perform some search queries. This article assumes no prior knowledge of Docker; just assumes that Docker is already installed. To install Docker, visit https://get.docker.com.

$ docker run -it -p 8983:8983 -p 9983:9983 solr:7.1 /opt/solr/bin/solr -c -f

At this point, Solr is started up and is ready to go (point your browser to http://localhost:8983/solr to view the Solr’s Admin UI). Before we proceed, though, let us understand the various parts of the Docker command used to start Solr. The base command is “run” which is used to instantiate a Docker container within which Solr will be running.

1. The flags “-it” instruct Docker to run the container interactively (as opposed to running it in the background) and allocating a pseudo TTY to go with it.
2. The flag “-p” is used to expose a port from within a container and map it to a port opened in the host computer (where Docker is running). Since 8983 is the default Solr port, it is exposed through this mechanism so that we could interact with Solr now. The port 9983 in this example refers to a ZooKeeper port to which other Solr containers can potentially connect to later so as to form a Solr cluster of multiple Solr nodes.
3. “solr:7.1” refers to the application and version that needs to be started. In this case, the Solr application’s official Docker image will be pulled from the central Docker repositories (called Docker Hub) and the image will be used to start Solr containers.
4. “/opt/solr/bin/solr -c -f” is the command that starts Solr after the container is started. Inside the container, Solr is installed in the /opt/solr directory and the ./bin/solr script is used to start Solr. The -c parameter to the bin/solr script instructs it to start Solr in “cloud” mode or “SolrCloud” mode. It means that Solr would start up as part of a cluster in a distributed mode. In the cloud mode, an (embedded) instance of ZooKeeper, used for cluster coordination, would be started up alongside the Solr process; other Solr nodes can be made to be part of this SolrCloud cluster by connecting themselves to this ZooKeeper instance. The “-f” parameter instructs the bin/solr script to start Solr in the foreground mode so that the Docker container continues to run and the logs are displayed.

Interacting with Solr: A books collection

From a separate terminal, issue the following commands:

Create a collection:

curl -X POST \
  http://localhost:8983/api/collections \
  -d '{
  "create": {
    "name": "books",
    "numShards": 1
  }
}'

Indexing documents into the collection:

Indexing one document at a time:

curl -X POST \
  -D '{"id":"1", "title":"Hitchhikers Guide to the Galaxy", "author":"Douglas Adams"}' \
  http://localhost:8983/api/collections/books/update?commit=true

curl -X POST \
  -D '{"id":"2", "title":"My Family and Other Animals", "author":"Gerald Durrell"}' \
  http://localhost:8983/api/collections/books/update?commit=true

curl -X POST \
  -D '{"id":"3", "title":"1984", "author":"George Orwell"}' \
  http://localhost:8983/api/collections/books/update?commit=true

curl -X POST \
  -D '{"id":"4", "title":"Lucene in Action", "author":"Erik Hatcher"}' \
  http://localhost:8983/api/collections/books/update?commit=true

curl -X POST -d \
  '[
    {"id":"1", "title":"Hitchhikers Guide to the Galaxy", "author":"Douglas Adams"},
    {"id":"2", "title":"My Family and Other Animals", "author":"Gerald Durrell"},
    {"id":"3", "title":"1984", "author":"George Orwell"},
    {"id":"4", "title":"Lucene in Action", "author":"Erik Hatcher"}
  ]' \
  "http://localhost:8983/api/collections/books/update?commit=true"

Search queries:

Get all Solr documents (books):

curl "http://localhost:8983/api/collections/books/select?q=*:*"

curl "http://localhost:8983/api/collections/books/select?q=title:lucene"

curl "http://localhost:8983/api/collections/books/select?q=author:orwell"

curl 'http://localhost:8983/api/collections/books/select?q=author:"douglas+adams"'

Conclusion

This was just a quick 5 minute introduction. There are various nuances associated with collection creation (like sharding, replication), indexing (schema management etc.) and querying (different query parsers etc.). We shall explore all of those aspects in subsequent parts. Refer to the official reference guide for more details.