Multithreading – File Encryption

The original main thread is not part of these three groups. The main() function should open the source file, and create/initialize a common buffer, and create all READER, WORKER, WRITER threads. Then, the main thread waits for all these threads to finish. All READER, WORKER and WRITER threads share the same buffer. Each slot in the common buffer stores 3 pieces of information: one data byte from the source file, its offset in the source file and a flag to indicate its status in the buffer, such as unprocessed, changing, processed, an so on.

typedef struct {
    char data;
    off_t offset;
    char state;   
} BufferItem;

Each READER thread goes to sleep for some random time between 0 and 0.01 seconds upon being created. Then, it reads the next single byte from the input file and saves that byte and its offset in the file to the next available empty slot in the buffer. Then, this READER threads goes to sleep for some random time between 0 and 0.01 seconds and then goes back to read the next byte of the file until the end of file. If the buffer is full, READER threads go to sleep for some random time between 0 and 0.01 seconds and then go back to check again.

Meanwhile, upon being created each WORKER thread sleeps for some random time between 0 and 0.01 seconds and it reads next byte in the buffer and process one byte of data, either encrypts or decrypt according to the working mode. Then the WORKER thread goes to sleep for some random time between 0 and 0.01 seconds and goes back to process next byte in the buffer until the entire file is done. If the buffer is empty, the WORKER threads go to sleep for some random time between 0 and 0.01 seconds and then go back to check again. If running in the encrypt mode, each WORKER thread will encrypt each data byte in the buffer, from original ASCII code to secret code for each character in the file, according to the following formula:

if (data > 31 && data < 127)
    data = (((int)data - 32) + 2 * 95 + KEY) % 95 + 32;

If running in the decrypt mode, each WORKER thread decrypts each data in the buffer, from secret code to original ASCII code, according to the following formula:

if (data > 31 && data < 127)
    data = (((int)data - 32) + 2 * 95 - KEY) % 95 + 32;

where KEY is a key value (between 0 and 127). If you use the same value for both encryption and decryption, it can perfectly recover the original data.

Similarly, upon being created, each WRITER thread sleeps for some random time between 0 and 0.01 seconds and it reads a processed byte and its offset from the next available nonempty buffer slot, and then writes the byte to that offset in the target file. Then, it also goes to sleep for some random time between 0 and 0.01 seconds and goes back to copy next byte until nothing is left. If the buffer is empty, the WRITER threads go to sleep for some random time between 0 and 0.01 seconds and then go back to check again.

Since all threads access common data, synchronization is required. The following pthread API’s are used:

Critical sections of code are used for READER, WORKER and WRITER threads.

As a simple debugging strategy, using the same KEY value to run the encryption.c to process any text file twice, one in encryption mode and one in decryption mode, the original text file will be perfectly recovered.

 Single Test Case

The following bash script does four things: 1) Compiles program, 2) Encrypts input file, 3) Decrypts encrypted file, and 4) Verifies file integrity

#!/usr/bin/env bash
#/*************************************************************
# * Author:             Victor Diaz
# * To run              chmod +x run.sh
# *                     ./run.sh       
# *                     ./run.sh encrypt.c 1 10 20 30 pokemon-small.txt 1024
# **************************************************************/

function print_usage {
    echo "Usage: $0       "
}

if [ "$#" -ne "7" ]; then
    print_usage $0
    exit -1
fi

#Compile the program
gcc $1 -o encrypt -lpthread

if [ "$?" -eq  "0" ]; then
    printf "  * Running encryption ...\n"
    printf "    Input File   : %s\n" $6
    printf "    Output File  : %s\n\n" $6.enc
    ./encrypt $2 $3 $4 $5 $6 $6.enc $7
else
    echo "Compilation failed... aborting program"
    exit -1
fi

if [ "$?" -eq  "0" ]; then
    printf "  * Running decryption ...\n"
    printf "    Input File   : %s\n" $6.enc
    printf "    Output File  : %s\n\n" $6.dec
    ./encrypt -$2 $3 $4 $5 $6.enc $6.dec $7
else
    echo "Encryption failed... aborting program"
    exit -1
fi

if [ "$?" -eq "0" ]; then
    printf "  * Checking file integrity ...\n"
    printf "    Input File A : %s\n" $6
    printf "    Input File B : %s\n" $6.dec
    diff -a $6 $6.dec
    if [ "$?" -eq "0" ]; then
        printf "    *** Success! Files are identical! ***\n"
    fi
    printf "\n"
fi

 Multiple Test Cases

With the help of the following bash script, we can include multiple test cases.

#!/usr/bin/env bash
#/*************************************************************
# * Author:             Victor Diaz
# * To run              chmod +x runall.sh
# *                     ./runall.sh 
# *                     ./runall.sh encrypt.c
# **************************************************************/

#EDIT THIS PART TO MATCH YOUR FILES
# * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * 
small_file="pokemon-small.txt"
big_file="pokemon-big.txt"
# * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * 

function print_usage {
    echo "Usage: $0 "
}

#run a test 
#PARAMS:        
function runTest {
    ./encrypt $1 $2 $3 $4 $5 "$5-$8.enc" $6

    if [ "$?" -eq "0" ]; then
        ./encrypt -$1 $2 $3 $4 "$5-$8.enc" "$5-$8.dec" $6

        if [ "$?" -eq "0" ]; then
            #echo -e "Checking result:\n"
            diff -a $5 "$5-$8.dec"
            if [ "$?" -eq "0" ]; then
                printf "%2d %-50s %s\n" $8 $7 "*** Success ***"
                return 2
            fi
        fi
    fi
    return 1
}

function printResult {
    printf "\n"
    printf "%s %s\n" "-----------------------------------------------------" "---------------"
    printf "%-53s %s\n" "Number of tests passed:" $1/\
    printf "%s %s\n" "-----------------------------------------------------" "---------------"
}

function main {
    numTests=0
    success=0

    printf "%s %s %s\n" "--" "--------------------------------------------------" "---------------"
    printf "%s %-50s %s\n" "id" "test-name" "test-status"
    printf "%s %s %s\n" "--" "--------------------------------------------------" "---------------"

    #----------------------------------------------------------
    # Small Tests
    #----------------------------------------------------------
    let "numTests++"
    runTest 1 1 1 1 $small_file 1 "small_file_basic" $numTests
    if [ "$?" -eq "2" ]; then 
        let "success++"
    fi

    let "numTests++"
    runTest 1 1 1 1 $small_file 100 "small_file_large_buffer" $numTests
    if [ "$?" -eq "2" ]; then 
        let "success++"
    fi

    let "numTests++"
    runTest 1 10 10 10 $small_file 1 "small_file_single_buffer_10_threads" $numTests
    if [ "$?" -eq "2" ]; then 
        let "success++"
    fi

    let "numTests++"
    runTest 1 10 10 10 $small_file 100 "small_file_large_buffer_10_threads" $numTests
    if [ "$?" -eq "2" ]; then 
        let "success++"
    fi

    #----------------------------------------------------------
    # Big Tests
    #----------------------------------------------------------
    let "numTests++"
    runTest 1 1 1 1 $big_file 200 "big_file_large_buffer" $numTests
    if [ "$?" -eq "2" ]; then 
        let "success++"
    fi

    let "numTests++"
    runTest 1 10 10 10 $big_file 10 "big_file_buffer_10_threads_10_each" $numTests
    if [ "$?" -eq "2" ]; then 
        let "success++"
    fi

    let "numTests++"
    runTest 1 100 5 5 $big_file 100 "big_file_big_buffer_100In_5Work_5Out" $numTests
    if [ "$?" -eq "2" ]; then 
        let "success++"
    fi

    let "numTests++"
    runTest 1 5 100 5 $big_file 100 "big_file_big_buffer_5In_100Work_5Out" $numTests
    if [ "$?" -eq "2" ]; then 
        let "success++"
    fi

    let "numTests++"
    runTest 1 5 5 100 $big_file 100 "big_file_big_buffer_5In_5Work_100Out" $numTests
    if [ "$?" -eq "2" ]; then 
        let "success++"
    fi

    let "numTests++"
    runTest 1 1000 1000 1000 $big_file 1000 "big_file_1000_all" $numTests
    if [ "$?" -eq "2" ]; then 
        let "success++"
    fi

    printResult $success $numTests
    if [ $success -eq $numTests ]; then
        echo "All the success in your life!!!!"
        echo ""
    fi
}

if [ "$#" -ne "1" ]; then
    print_usage $0
    exit -1
fi

#Compile the program
gcc -W -Wall $1 -o encrypt -lpthread 

if [ "$?" -eq "0" ]; then
    echo "Compilation complete... running encryption/decryption tests"
    main $@
else
    echo "Compilation failed... aborting program"
    exit -1
fi

 API

 References

1. The C Programming Language, 2nd Edition
   Brian W. Kernighan, Dennis Ritchie
2. Beej's Guide to Unix IPC
   Brian "Beej Jorgensen" Hall
3. Beej's Guide to Unix IPC - PDF
   Brian "Beej Jorgensen" Hall
4. Operating System Concepts, 8th Edition
   Abraham Silberschatz, Peter B. Galvin, Greg Gagne
5. Linux man pages
   Michael Kerrisk