Creating Partial Cloudant Indexes

Filter data before it’s indexed to get the most out of your storage

Indexing is what makes database queries fast and scalable. Without an index, a database is forced to trawl through every record to calculate the answer to a query. A carefully designed index allows a query to be answered with a fraction of a work by jumping to the pertinent portion.

There are 1,176,672 records in the database, so flying without an index is going to be an expensive operation. In fact, if you attempt to make a query without first creating an index, Cloudant will oblige but issue a warning in the returned data:

The fields you need to index are related to the fields that you use in the selector object in your query. If we are going to be making queries between two dates, then it makes sense to create an index on the Date field:

In this case we only need to index the Date field.

Once the index is built, we can then query this index, passing in a date or a range of dates, and Cloudant will use the index to fulfill the request:

The first index we created stored an entry for each of the 1m+ records. This may be exactly what we intended, but in some cases we may only ever be interested in a subset of the data.

At query-time, you can further winnow the data on the fields you chose to index:

Our query for 2015 accidents using this partial index now only contains accidents that occurred during the weekend.

For extra code readability, you can also include the original partial selector in your query-time selector:

Partial indexes pre-filter the data before it is written to the index. This can make your indicies smaller, leaner and quicker for Cloudant to work with. Also, as Cloudant pay-as-you-go services charge per GB of data (which includes the data used to store your core JSON and index data), smaller indexes can help keep your costs down too!

Related posts:

Financial engineering is the application of mathematics to financial questions, typically using computer models If we asked you to calculate the price of a financial option, you’d probably say you’ll…

XZEN Project is happy to provide you with the first Monthly Progress Report December 2017. The following milestones have been completed by XZEN Project team within the reporting month: 5. Conclusion…

2018 is all about contrast: blending vectors with photographs, retro patterns with modern colors and typography, two-dimensional environments with three-dimensional designs, and static illustrations…