One of the things that concerns most users of public cloud is about the security of their data. In this post, we’re going to talk about how can secure the data in your Azure Cognitive Search Service Indexes and Synonym Maps by using proper encryption and how Cerulean makes it super easy to set this up.
First, let’s talk about data encryption. Well, the good news is that by default the data contained in your Azure Cognitive Search Service Indexes and Synonym Maps is stored in encrypted format (this is commonly known as “Encryption at Rest”). You don’t have to do anything special to accomplish this. All the data stored is encrypted using encryption keys provided by Microsoft. Data encryption/decryption happens transparently without you having to do anything.
The main thing to notice above is that the data is encrypted by default using keys provided by Microsoft. However, consider a scenario where your organization would want to encrypt the data using your own key. Again, the good news is that it is entirely possible to do so however there are some things you would need to do to make it working. In this post we will talk about how you can accomplish it and how Cerulean makes it easy for you.
The process where you (a customer) want to encrypt the data using your own encryption keys is called “Customer-managed encryption”. Broadly the way it works is that You tell Azure Cognitive Search Service that you would like to use your own key. Once it is configured properly, Azure Cognitive Search would use that key for encryption/decryption of data. Again, the encryption/decryption happens transparently thus you don’t have to write any special code in your application to perform encryption/decryption.
Azure Key Vault
Azure requires that you store your encryption key in Azure Key Vault. You can create your own keys and store them in Azure Key Vault or you can directly create a key there using Azure Portal, Azure CLI/PowerShell, Azure SDKs or directly using Azure Key Vault REST API.
For the sake of brevity, the instructions for creating the key is not mentioned here in this post. You can follow the instructions described in the following to do the same: https://docs.microsoft.com/en-us/azure/search/search-security-manage-encryption-keys (Steps 1 & 2). However, in short here’s what you would need to do:
- Create a Key Vault if not done already.
- Create a key with all permissions and copy the key identifier.
Azure Active Directory (AD) Application
Next step would be to create an Azure AD application which will have permission to read the key you created from Azure Key Vault.
Again for the sake of brevity, the instructions for creating and configuring Azure AD application is not mentioned here in this post. You can follow the instructions described in the following to do the same: https://docs.microsoft.com/en-us/azure/search/search-security-manage-encryption-keys (Steps 3 & 4).
Instead of creating an Azure AD application, you can also configure your Azure Cognitive Search Service as a “Trusted Service” by enabling a system-managed identity for it. You would then use the trusted search service as a security principle, rather than an AD-registered application, to access the key vault key.
Once you’re done with these two steps, you should have the following:
- Key identifier of the key you created.
- Azure AD application identifier and a client secret or have configured Search Service as a Trusted Service.
Now you’re ready to protect the contents of your Azure Cognitive Search Index using your key.
Cerulean (a cross-platform desktop tool to manage your Azure resources) makes it super easy to manage the encryption of data in your Azure Cognitive Search Index. It is the only tool that provides most comprehensive set of features to manage your Azure Cognitive Search resources (Indexes, Data Sources, Indexers, Synonym Maps, Data etc.). You can learn more about Azure Cognitive Search features in Cerulean on our website at https://cerebrata.com/products/cerulean/features/azure-search.
Please note that Cerulean is the only tool that enables you to manage encryption through an intuitive user interface. For simple index definitions, you can certainly consume REST API directly however as your Index definition becomes somewhat complicated (e.g. if you wish to add things like Char Filters, Tokenizers, Token Filters and Analyzers), it is recommended that you use a tool like Cerulean.
Let’s say you are creating a new Index and would like to encrypt the contents of that index. To do so in Cerulean, please follow the steps mentioned below:
- Start the process of creating a new Index.
- When the tab for creating a new Index opens up, provide a name for your Index.
- Click on “Encryption” button as shown below. You could also use Control+K (Windows/Linux) or ⌘+K (Mac) keyboard shortcut.
- Clicking on this button will open up a popup as shown below.
- Encryption Option: This is where you select either “Microsoft” managed keys or “Customer” managed keys. Since we’re interested in the latter, we will choose that option.
- Key Vault URI: This is the URI of your Azure Key Vault from the “Key Identifier”. For example, if you Key Identifier is something like “https://demokeyvault.vault.azure.net/keys/myEncryptionKey/eaab6a663d59439ebb95ce2fe7d5f660”, your Key Vault URI will be “https://demokeyvault.vault.azure.net”.
- Key Vault Key Name: This is the name of the key. Using the example above, the key name would be “myEncryptionKey”.
- Key Vault Key Version: This is the version of the key. Again, using the example above, the key version would be “eaab6a663d59439ebb95ce2fe7d5f660”.
- Identity Option: This is where you select whether you have configured your Azure Cognitive Search Service as a Trusted Service or have created an Azure AD application. It is recommended that you go with former instead of latter.
- Application Id: In case you have created an Azure AD application (i.e. opted for latter option), this is the identifier of the application you created in Azure AD.
- Application Secret: This is the secret you created for your Azure AD application.
- OK: Once you have specified all settings, simply click OK. Cerulean will give you warning if you missed out providing the information in any of the required fields.
- Proceed with setting up other properties of the Index like Fields, Scoring Profiles, Suggesters, CORS, Char Filters, Tokenizers, Token Filters and Analyzers.
- Once you’re done with setting up all properties, simply click on “Save Index” button to create the Index in your Azure Cognitive Search Service account.
Congratulations! Now the data in your Index is encrypted/decrypted using your key.
Now let’s consider the scenario where you have an existing index and you wish to change some aspects of encryption/decryption. The process would remain exactly the same in the sense that you will open up an Index for editing (Manage Index Settings) and then click on “Encryption” button.
However, please keep in mind that you cannot change the “Encryption” option. What that means is that if you created an Index with “Microsoft-managed keys” encryption option, unfortunately you can’t change it to “Customer-managed keys” encryption option. The only option available to you would be to delete that index and recreate a new one with “Customer-managed keys” encryption option.
Thus if you opted for “Microsoft-managed keys” encryption option, you can’t really change anything. However if you chose “Customer-managed keys” encryption option while creating the Index, you can change the settings.
For Synonym Maps, the process is exactly identical to that of managing encryption in an Index.
In this post we learnt about how you can protect the data stored in your Azure Cognitive Search Service Indexes and Synonym Maps by encrypting it using your own key stored in Azure Key Vault. We also learnt about how Cerulean makes it really easy for you to do that.
Apart from Azure Cognitive Search, Cerulean also has support for some other Azure services like Cosmos DB, Storage, Service Bus, Redis Cache etc. We strongly encourage you to try out Cerulean for managing your Azure resources. You can find more information about all the features in Cerulean on our website at https://www.cerebrata.com.