PHP Classes

Cipher Sweet: Encrypt data in away that can be searched

Recommend this page to a friend!
     
  Info   Example   View files Files   Install with Composer Install with Composer   Download Download   Reputation   Support forum   Blog    
Ratings Unique User Downloads Download Rankings
Not enough user ratingsTotal: 99 All time: 9,819 This week: 53Up
Version License PHP version Categories
ciphersweet 1.0MIT/X Consortium ...5PHP 5, Databases, Cryptography, Security
Description 

Author

This package can encrypt data in away that can be searched.

It can take string of data and a key to create an encrypted version of the data and indexes that can be stored for instance in a database.

Applications can search for the values looking up for index values. Then they can decrypt the data using this package.

If multiple records are found to match the same index value, the applications can traverse a smaller list of records that were found during the search to find the stored records that have the exact value of the key they are searching for.

Innovation Award
PHP Programming Innovation award nominee
September 2018
Number 4
One challenge of applications that need to encrypt information to prevent it to been seen by untrusted parties is that when that information needs to be searched, there is the need to look at the decrypted information to find the relevant records of data.

This package implements a solution that creates indexes before encrypting the information, so the searching components can look at the indexes first before actually decrypting a smaller set of possible records that may contain the information is being searched.

This way the search component does not need to decrypt all database records to check which records contain the information that applications need to find.

Manuel Lemos
Picture of Scott Arciszewski
  Performance   Level  
Name: Scott Arciszewski <contact>
Classes: 37 packages by
Country: United States United States
Age: ???
All time rank: 1172170 in United States United States
Week rank: 136 Up14 in United States United States Up
Innovation award
Innovation award
Nominee: 28x

Winner: 1x

Example

Using CipherSweet

Setting up CipherSweet at Run-Time

Select Your Backend

First, you'll need to decide if you have any strict operational requirements for your encryption. This mostly boils down to whether or not you need all encryption to be FIPS 140-2 compliant or not, in which case, you'll need to use the FIPSCrypto backend.

If you aren't sure, the answer is that you probably don't, and feel free to use ModernCrypto instead.

<?php
use ParagonIE\CipherSweet\Backend\FIPSCrypto;
use ParagonIE\CipherSweet\Backend\ModernCrypto;

$fips = new FIPSCrypto(); // Use only FIPS 140-2 algorithms
$nacl = new ModernCrypto(); // Uses libsodium

Define your Key Provider

After you choose your backend, you'll need a KeyProvider. We provide a few out-of-the-box, but we also provide an interface that can be used to integrate with any key management service in your code.

The simplest example of this is the StringProvider, which accepts a string containing your encryption key:

<?php
use ParagonIE\ConstantTime\Hex;
use ParagonIE\CipherSweet\Backend\ModernCrypto;
use ParagonIE\CipherSweet\KeyProvider\StringProvider;

$provider = new StringProvider(
    new ModernCrypto(),
    // Example key, chosen randomly, hex-encoded:
    '4e1c44f87b4cdf21808762970b356891db180a9dd9850e7baf2a79ff3ab8a2fc'
);

You can pass a raw binary string, hex-encoded string, or base64url-encoded string to the second parameter of the StringProvider constructor, provided the decoded key is 256 bits.

Attempting to pass a key of an invalid size (i.e. not 256-bit) will result in a CryptoOperationException being thrown. The recommended way to generate a key is:

<?php
use ParagonIE\ConstantTime\Hex;

var_dump(Hex::encode(random_bytes(32)));

Start Your Engines

Once you have these two, you can actually start the engine (CipherSweet). Building on the previous code example:

<?php
use ParagonIE\ConstantTime\Hex;
use ParagonIE\CipherSweet\Backend\FIPSCrypto;
use ParagonIE\CipherSweet\CipherSweet;
use ParagonIE\CipherSweet\KeyProvider\StringProvider;

$provider = new StringProvider(
    new ModernCrypto(),
    // Example key, chosen randomly, hex-encoded:
    '4e1c44f87b4cdf21808762970b356891db180a9dd9850e7baf2a79ff3ab8a2fc'
);

$engine = new CipherSweet($provider);

If you're using FIPSCrypto instead of ModernCrypto, you just need to pass it once to the KeyProvider and the rest is handled for you.

<?php
use ParagonIE\ConstantTime\Hex;
use ParagonIE\CipherSweet\Backend\ModernCrypto;
use ParagonIE\CipherSweet\CipherSweet;
use ParagonIE\CipherSweet\KeyProvider\StringProvider;

$provider = new StringProvider(
    new FIPSCrypto(),
    // Example key, chosen randomly, hex-encoded:
    '4e1c44f87b4cdf21808762970b356891db180a9dd9850e7baf2a79ff3ab8a2fc'
);
$engine = new CipherSweet($provider);

Basic CipherSweet Usage

Once you have an engine in play, you can start defining encrypted fields and defining one or more blind index to be used for fast search operations.

EncryptedField

This will primarily involve the EncryptedField class (as well as one or more instances of BlindIndex), mostly:

  • `$encryptedField->prepareForStorage()`
  • `$encryptedField->getBlindIndex()`
  • `$encryptedField->getAllBlindIndexes()`
  • `$encryptedField->encryptValue()`
  • `$encryptedField->decryptValue()`

For example, the following code encrypts a user's social security number and then creates two blind indexes: One for a literal search, the other only matches the last 4 digits.

<?php
use ParagonIE\CipherSweet\BlindIndex;
use ParagonIE\CipherSweet\CipherSweet;
use ParagonIE\CipherSweet\EncryptedField;
use ParagonIE\CipherSweet\Transformation\LastFourDigits;

/ @var CipherSweet $engine */
$ssn = (new EncryptedField($engine, 'contacts', 'ssn'))
    // Add a blind index for the "last 4 of SSN":
    ->addBlindIndex(
        new BlindIndex(
            // Name (used in key splitting):
            'contact_ssn_last_four',
            // List of Transforms: 
            [new LastFourDigits()],
            // Bloom filter size (bits)
            16
        )
    )
    // Add a blind index for the full SSN:
    ->addBlindIndex(
        new BlindIndex(
            'contact_ssn', 
            [],
            32
        )
    );

// Some example parameters:
$contactInfo = [
    'name' => 'John Smith',
    'ssn' => '123-45-6789',
    'email' => 'foo@example.com'
];

/ 
 * @var string $ciphertext
 * @var array<string, string> $indexes
 */
list ($ciphertext, $indexes) = $ssn->prepareForStorage($contactInfo['ssn']);

Every time you run the above code, the $ciphertext will be randomized, but the array of blind indexes will remain the same.

Each blind index returns an array with two values: type and value. The value is calculated from the plaintext. The type is a key derived form the table name, field name, and index name.

The type indicator is handy if you're storing all your blind indexes in a separate table rather than in an additional column in the same table. In the latter case, you only need the value string for each index.

var_dump($ciphertext, $indexes);
/*
string(73) "nacl:jIRj08YiifK86YlMBfulWXbatpowNYf4_vgjultNT1Tnx2XH9ecs1TqD59MPs67Dp3ui"
array(2) {
  ["contact_ssn_last_four"]=>
  array(2) {
    ["type"]=>
    string(13) "3dywyifwujcu2"
    ["value"]=>
    string(4) "8058"
  }
  ["contact_ssn"]=>
  array(2) {
    ["type"]=>
    string(13) "2iztg3wbd7j5a"
    ["value"]=>
    string(8) "311314c1"
  }
}
*/

You can now use these values for inserting/updating records into your database.

To search the database at a later date, use getAllBlindIndexes() or getBlindIndex():

<?php
use ParagonIE\CipherSweet\BlindIndex;
use ParagonIE\CipherSweet\CipherSweet;
use ParagonIE\CipherSweet\EncryptedField;
use ParagonIE\CipherSweet\Transformation\LastFourDigits;

/ @var CipherSweet $engine */
$ssn = (new EncryptedField($engine, 'contacts', 'ssn'))
    // Add a blind index for the "last 4 of SSN":
    ->addBlindIndex(
        new BlindIndex(
            // Name (used in key splitting):
            'contact_ssn_last_four',
            // List of Transforms: 
            [new LastFourDigits()],
            // Bloom filter size (bits)
            16
        )
    )
    // Add a blind index for the full SSN:
    ->addBlindIndex(
        new BlindIndex(
            'contact_ssn', 
            [],
            32
        )
    );

// Use these values in search queries:
$indexes = $ssn->getAllBlindIndexes('123-45-6789');
$lastFour = $ssn->getBlindIndex('123-45-6789', 'contact_ssn_last_four');

Which should result in the following (for the example key):

var_dump($lastFour);
/*
array(2) {
  ["type"]=>
  string(13) "3dywyifwujcu2"
  ["value"]=>
  string(4) "8058"
}
*/

var_dump($indexes);
/*
array(2) {
  ["contact_ssn_last_four"]=>
  array(2) {
    ["type"]=>
    string(13) "3dywyifwujcu2"
    ["value"]=>
    string(4) "8058"
  }
  ["contact_ssn"]=>
  array(2) {
    ["type"]=>
    string(13) "2iztg3wbd7j5a"
    ["value"]=>
    string(8) "311314c1"
  }
}
*/

EncryptedRow

An alternative approach for datasets with multiple encrypted rows and/or encrypted boolean fields is the EncryptedRow API, which looks like this:

<?php
use ParagonIE\CipherSweet\BlindIndex;
use ParagonIE\CipherSweet\CipherSweet;
use ParagonIE\CipherSweet\CompoundIndex;
use ParagonIE\CipherSweet\EncryptedRow;
use ParagonIE\CipherSweet\Transformation\LastFourDigits;

/ @var CipherSweet $engine */
// Define two fields (one text, one boolean) that will be encrypted
$row = (new EncryptedRow($engine, 'contacts'))
    ->addTextField('ssn')
    ->addBooleanField('hivstatus');

// Add a normal Blind Index on one field:
$row->addBlindIndex(
    'ssn',
    new BlindIndex(
        'contact_ssn_last_four',
        [new LastFourDigits()],
        32 // 32 bits = 4 bytes
    )
);

// Create/add a compound blind index on multiple fields:
$row->addCompoundIndex(
    (
        new CompoundIndex(
            'contact_ssnlast4_hivstatus',
            ['ssn', 'hivstatus'],
            32, // 32 bits = 4 bytes
            true // fast hash
        )
    )->addTransform('ssn', new LastFourDigits())
);

// Notice: You're passing an entire array at once, not a string
$prepared = $row->prepareRowForStorage([
    'extraneous' => true,
    'ssn' => '123-45-6789',
    'hivstatus' => false
]);

var_dump($prepared);
/*
array(2) {
  [0]=>
  array(3) {
    ["extraneous"]=>
    bool(true)
    ["ssn"]=>
    string(73) "nacl:wVMElYqnHrGB4hU118MTuANZXWHZjbsd0uK2N0Exz72mrV8sLrI_oU94vgsWlWJc84-u"
    ["hivstatus"]=>
    string(61) "nacl:ctWDJBn-NgeWc2mqEWfakvxkG7qCmIKfPpnA7jXHdbZ2CPgnZF0Yzwg="
  }
  [1]=>
  array(2) {
    ["contact_ssn_last_four"]=>
    array(2) {
      ["type"]=>
      string(13) "3dywyifwujcu2"
      ["value"]=>
      string(8) "805815e4"
    }
    ["contact_ssnlast4_hivstatus"]=>
    array(2) {
      ["type"]=>
      string(13) "nqtcc56kcf4qg"
      ["value"]=>
      string(8) "cbfd03c0"
    }
  }
}
*/

With the EncryptedRow API, you can encrypt a subset of all of the fields in a row, and create compound blind indexes based on multiple pieces of data in the dataset rather than a single field, without writing a ton of glue code.

Using CipherSweet with a Database

CipherSweet is database-agnostic, so you'll need to write some code that uses CipherSweet behind-the-scenes to encrypt data before storing it in a database, query the database based on blind indexes, and then use CipherSweet to decrypt the results.

See also: the examples directory.

Solutions for Common Problems with Searchable Encryption

See also: the solutions directory.


Details

CipherSweet

Linux Build Status Latest Stable Version Latest Unstable Version License Downloads

CipherSweet is a backend library developed by Paragon Initiative Enterprises for implementing searchable field-level encryption.

Requires PHP 5.5+, although 7.2 is recommended for better performance.

Before adding searchable encryption support to your project, make sure you understand the appropriate threat model for your use case. At a minimum, you will want your application and database server to be running on separate cloud instances / virtual machines. (Even better: Separate bare-metal hardware.)

CipherSweet is available under the very permissive ISC License which allows you to use CipherSweet in any of your PHP projects, commercial or noncommercial, open source or proprietary, at no cost to you.

CipherSweet Features at a Glance

  • Encryption that targets the 256-bit security level (using AEAD modes with extended nonces to minimize users' rekeying burden).
  • Compliance-Specific Protocol Support. Multiple backends to satisfy a diverse range of compliance requirements. More can be added as needed: * `ModernCrypto` uses libsodium, the de facto standard encryption library for software developers. * `FIPSCrypto` only uses the cryptographic algorithms covered by the FIPS 140-2 recommendations to avoid auditing complexity.
  • Key separation. Each column is encrypted with a different key, all of which are derived from your master encryption key using secure key-splitting algorithms.
  • Key management integration. CipherSweet supports integration with Key Management solutions for storing and retrieving the master encryption key.
  • Searchable Encryption. CipherSweet uses blind indexing with the fuzzier and Bloom filter strategies to allow fast ciphertext search with minimal data leakage. * Each blind index on each column uses a distinct key from your encryption key and each other blind index key.
  • Adaptability. CipherSweet has a database- and product-agnostic design, so it should be easy to write an adapter to use CipherSweet in any PHP-based software.

Installing CipherSweet

Use Composer.

composer require paragonie/ciphersweet

Using CipherSweet

Please refer to the documentation to learn how to use CipherSweet.

Integration Support

Please feel free to create an issue if you'd like to integrate CipherSweet with your software.

Why "CipherSweet"?

CipherSweet was originally intend for use in in SuiteCRM (a fork of the SugarCRM Community Edition) and related products, although there is nothing preventing its use in other products.

Therefore, we opted for a pun on "ciphersuite" that pays homage to the open source heritage of the project we designed this library for.

If the wordplay is too heavy, feel free to juxtapose the two component nouns and call it "SweetCipher" in spoken conversation.


  Files folder image Files (63)  
File Role Description
Files folder imagedocs (1 file, 3 directories)
Files folder imagesrc (6 files, 5 directories)
Files folder imagetests (6 files, 3 directories)
Accessible without login Plain text file .travis.yml Data Auxiliary data
Accessible without login Plain text file composer.json Data Auxiliary data
Accessible without login Plain text file LICENSE Lic. License text
Accessible without login Plain text file phpunit.xml.dist Data Auxiliary data
Accessible without login Plain text file psalm.xml Data Auxiliary data
Accessible without login Plain text file README.md Doc. Documentation

  Files folder image Files (63)  /  docs  
File Role Description
Files folder imageexamples (5 files)
Files folder imageinternals (6 files)
Files folder imagesolutions (2 files)
  Accessible without login Plain text file README.md Example Example script

  Files folder image Files (63)  /  docs  /  examples  
File Role Description
  Accessible without login Plain text file 01-easydb-latitude.md Example Example script
  Accessible without login Plain text file 02-pdo-mysql.md Example Example script
  Accessible without login Plain text file 03-doctrine-mysql.md Class Class source
  Accessible without login Plain text file 04-key-generation.md Data Auxiliary data
  Accessible without login Plain text file README.md Doc. Documentation

  Files folder image Files (63)  /  docs  /  internals  
File Role Description
  Accessible without login Plain text file 01-key-heirarchy.svg Data Auxiliary data
  Accessible without login Plain text file 01-key-hierarchy.md Data Auxiliary data
  Accessible without login Plain text file 02-packing.md Data Auxiliary data
  Accessible without login Plain text file 03-encryption.md Data Auxiliary data
  Accessible without login Plain text file 04-blind-index.md Data Auxiliary data
  Accessible without login Plain text file README.md Doc. Documentation

  Files folder image Files (63)  /  docs  /  solutions  
File Role Description
  Accessible without login Plain text file 01-boolean.md Example Example script
  Accessible without login Plain text file README.md Doc. Documentation

  Files folder image Files (63)  /  src  
File Role Description
Files folder imageBackend (2 files, 1 directory)
Files folder imageContract (3 files)
Files folder imageException (7 files)
Files folder imageKeyProvider (4 files)
Files folder imageTransformation (4 files)
  Accessible without login Plain text file BlindIndex.php Class Class source
  Accessible without login Plain text file CipherSweet.php Class Class source
  Accessible without login Plain text file CompoundIndex.php Class Class source
  Accessible without login Plain text file EncryptedField.php Class Class source
  Accessible without login Plain text file EncryptedRow.php Class Class source
  Accessible without login Plain text file Util.php Class Class source

  Files folder image Files (63)  /  src  /  Backend  
File Role Description
Files folder imageKey (1 file)
  Accessible without login Plain text file FIPSCrypto.php Class Class source
  Accessible without login Plain text file ModernCrypto.php Class Class source

  Files folder image Files (63)  /  src  /  Backend  /  Key  
File Role Description
  Accessible without login Plain text file SymmetricKey.php Class Class source

  Files folder image Files (63)  /  src  /  Contract  
File Role Description
  Accessible without login Plain text file BackendInterface.php Class Class source
  Accessible without login Plain text file KeyProviderInterface.php Class Class source
  Accessible without login Plain text file TransformationInterface.php Class Class source

  Files folder image Files (63)  /  src  /  Exception  
File Role Description
  Accessible without login Plain text file ArrayKeyException.php Class Class source
  Accessible without login Plain text file BlindIndexNameCollisionException.php Class Class source
  Accessible without login Plain text file BlindIndexNotFoundException.php Class Class source
  Accessible without login Plain text file CipherSweetException.php Class Class source
  Accessible without login Plain text file CryptoOperationException.php Class Class source
  Accessible without login Plain text file InvalidCiphertextException.php Class Class source
  Accessible without login Plain text file KeyProviderException.php Class Class source

  Files folder image Files (63)  /  src  /  KeyProvider  
File Role Description
  Accessible without login Plain text file ArrayProvider.php Class Class source
  Accessible without login Plain text file FileProvider.php Class Class source
  Accessible without login Plain text file RandomProvider.php Class Class source
  Accessible without login Plain text file StringProvider.php Class Class source

  Files folder image Files (63)  /  src  /  Transformation  
File Role Description
  Accessible without login Plain text file Compound.php Class Class source
  Accessible without login Plain text file DigitsOnly.php Class Class source
  Accessible without login Plain text file LastFourDigits.php Class Class source
  Accessible without login Plain text file Lowercase.php Class Class source

  Files folder image Files (63)  /  tests  
File Role Description
Files folder imageBackend (2 files)
Files folder imageKeyProvider (3 files)
Files folder imageTransformation (5 files)
  Accessible without login Plain text file BlindIndexTest.php Class Class source
  Accessible without login Plain text file CipherSweetTest.php Class Class source
  Accessible without login Plain text file CompoundIndexTest.php Class Class source
  Accessible without login Plain text file EncryptedFieldTest.php Class Class source
  Accessible without login Plain text file EncryptedRowTest.php Class Class source
  Accessible without login Plain text file UtilTest.php Class Class source

  Files folder image Files (63)  /  tests  /  Backend  
File Role Description
  Accessible without login Plain text file FIPSCryptoTest.php Class Class source
  Accessible without login Plain text file ModernCryptoTest.php Class Class source

  Files folder image Files (63)  /  tests  /  KeyProvider  
File Role Description
  Accessible without login Plain text file ArrayProviderTest.php Class Class source
  Accessible without login Plain text file FileProviderTest.php Class Class source
  Accessible without login Plain text file StringProviderTest.php Class Class source

  Files folder image Files (63)  /  tests  /  Transformation  
File Role Description
  Accessible without login Plain text file CompoundTest.php Class Class source
  Accessible without login Plain text file DigitsOnlyTest.php Class Class source
  Accessible without login Plain text file GenericTransfomationCase.php Class Class source
  Accessible without login Plain text file LastFourDigitsTest.php Class Class source
  Accessible without login Plain text file LowercaseTest.php Class Class source

The PHP Classes site has supported package installation using the Composer tool since 2013, as you may verify by reading this instructions page.
Install with Composer Install with Composer
 Version Control Unique User Downloads Download Rankings  
 100%
Total:99
This week:0
All time:9,819
This week:53Up