Evoluator

Guys I need some serious HELP (S.O.S)

I am creating a Cryptography engine which encrypts and decryptes files. I would like to use packets for encryption, so not all of the file would be loaded onto the memory. This is because the files are about 5GB in size, and even if I try to load all the file on to my computers memory my computer would soon run out of resources.

I have had this problem for about 2 weeks now, and I haven't found a good way of achieving this! I mean when I encrypt a file everything goes fine, but when decryption is taking place the original file in not there any more, it only has random values!!!!


Thanks for the help



Re: Visual Basic Express Edition Cryptography of packets!

ReneeC

I'd help you wth this Evoluator, but I'm not a guy.






Re: Visual Basic Express Edition Cryptography of packets!

Evoluator

lol!

That is a good way of looking at it. But I implicitly meant, mates (or rather, friends).


I'll rephrase:

Dears could you please help me with the question asked above Wink





Re: Visual Basic Express Edition Cryptography of packets!

ReneeC

Wahoo,

I will do my best..... but I think we need to talk at length. Do you have an MSN address If so post it and I'll contact you.

there are lot's of tricks to this. I wrote my own encryption algorith. It's less than brilliant but I believe you'd have to be NSA to decrypt it. I also have it in C++ which may not help you a lot. (It's from VS6). But I used it last night on a half gigabyte file and it worked fine (although I watched the temperature climb on the CPU).






Re: Visual Basic Express Edition Cryptography of packets!

Evoluator

I have MSN address and it is "My_live_ID@hotmail.co.uk".





Re: Visual Basic Express Edition Cryptography of packets!

Derek Smyth

Hi,

If the file is mostly text then you could try compressing the file down using the GZipStream class. The other option might be to read from the stream in blocks, but that will depend on the encryption method used and the mode of operation, also, for example you might need to read in two block to encrypt one.

This is a very interesting problem. I'd be very interested in how you solved this.






Re: Visual Basic Express Edition Cryptography of packets!

Evoluator

Well I have tried fe wmethods before asking the question.

One was to read certain blocks of say 4096 bytes at a time.{

A as integer = 4096 'Buffer size

Then I would encrypt this file and appended the data onto the destination file.

Only when the size to be read is less than a chunk of 4096 then the algorithm would read all of the remaining bytes
encrypt it, then converts is to Base64 then adds "@" to the encrypted chunk until the size reaches 4096


}


This method comes with lots of down sides! Sad(

Other methods have problems too, so I don't want to post it!


I am looking for something which could encrypt files of the size up to 100GB.

Also

an On The Fly encryption engine.








Re: Visual Basic Express Edition Cryptography of packets!

ReneeC

My approach was like evoluator's. This is not a hard problem. You just have to understand that you can't have all the data in memory at once to process and I though of it as frames. My algorithm is a shared key algorith where there is an external public key and the algorith is constantly get data from the system clock as a seed for that record. So the file increases in size by a small percentage. Since I was in C++ a used inline assembly to do byte rotations etc.

I don't pretent that this is a brilliant algorithm. I just don't think anyone could decode this unless they were sure it was a note from Osama Bin laden. Evoluator if you look on msn I'll be there soon.

"I am looking for something which could encrypt files of the size up to 100GB."

As far as I know, my algorithm is limitted by the size of storage.






Re: Visual Basic Express Edition Cryptography of packets!

Derek Smyth

I'm working on a solution now that should hopefully work, hopefully!. I think the important aspect of this is the mode of operation that the encryption method uses. This is how an encrypted block affects the next encrypted block. How you decrypt in 'packets' I believe will be controlled by this mode of operation. For example in Electronic Code Book mode one block doesn't effect the other block so I'm hoping (going to try this now) that the decryption of the packets should be fairly straight forward. I'll post the code if I get it working, and if I don't I shall delete this comment and write another one that doesn't pretend I know what I'm doing.

There is a tool called TrueCrypt that lets you create encrypted files that act like drives. Have a look at that and see if it covers what you need. www.truecrypt.org. It's freeeeee.






Re: Visual Basic Express Edition Cryptography of packets!

Derek Smyth

Heres an attempt.... using ECB, where one block in the encryption process doesn't effect the next encryption block. It works with a 83 Mb file but its was sluggish.

DESCryptoServiceProvider DES = new DESCryptoServiceProvider();

DES.BlockSize = 64;

DES.Mode = CipherMode.ECB;

The encryption process just loops over the incoming stream reading in 256 bytes at a time and encrypting it to an outgoing crypt stream. Important thing to note is it only writes out the bytes that was read in (amountReadEnc), for example there might only be 56 bytes in the last block so only write those 56 bytes.

///ENCRYPTION.....

//encrypt the large data set in blocks

byte[] plainBlock = new byte[256]; //des works in blocks of 8 bytes (do 256 bytes at a time)

//create a stream to a particularly large file (83Mb test)

FileStream plainStrmIn = new FileStream("C:\\test.zip", FileMode.Open, FileAccess.Read);

FileStream cipherStrmOut = new FileStream("C:\\test.enc", FileMode.Create, FileAccess.Write);

CryptoStream cryptoStrmEnc = new CryptoStream(cipherStrmOut, DES.CreateEncryptor(), CryptoStreamMode.Write);

int amountReadEnc = -1;

while (amountReadEnc != 0)

{

amountReadEnc = plainStrmIn.Read(plainBlock, 0, 256);

cryptoStrmEnc.Write(plainBlock, 0, amountReadEnc); //only write the bytes read from the stream

plainBlock = new byte[256]; //zero the plain block

}

cryptoStrmEnc.Close();

plainStrmIn.Close();

In the decryption method it does virtually the same thing in reverse, this was only possible because of ECB mode that was used. This is why I think the mode is important, if the mode had been CBC then the IV would have needed to been set.

//DECRYPTION

FileStream cipherStrmIn = new FileStream("C:\\test.enc", FileMode.Open, FileAccess.Read);

FileStream plainStrmOut = new FileStream("C:\\testDecrypt.zip", FileMode.Create, FileAccess.Write);

CryptoStream cryptoStrmDec = new CryptoStream(plainStrmOut, DES.CreateDecryptor(), CryptoStreamMode.Write);

byte[] cipherBlock = new byte[256]; //zero the plain block

int amountReadDec = -1;

while (amountReadDec != 0)

{

amountReadDec = cipherStrmIn.Read(cipherBlock, 0, 256);

cryptoStrmDec.Write(cipherBlock, 0, amountReadDec);

cipherBlock = new byte[256]; //zero the plain block

}

cryptoStrmDec.FlushFinalBlock();

cryptoStrmDec.Close();

cipherStrmIn.Close();

Creating your own approach is pretty cool ReneeC, encryption is just shuffling bits and I think it's ok to create your own if you keep the algorithm secret. We created one here because we had to interop between .NET and FORTRAN, which doesn't know any standard encryption mechanisms. The approach we took is crackable, no doubt, but it's not used to store any seriously important data for any length of time.






Re: Visual Basic Express Edition Cryptography of packets!

ReneeC

True confessions.....

I used to LOVE Fortran

I used to work for Digital used it as a shell to call system services. Used that way Fortran was a pretty good systems programming language.






Re: Visual Basic Express Edition Cryptography of packets!

Derek Smyth

Tried the code I posted yesterday out this morning in Cipher Block Chaining (CBC) mode and it worked without any changes, which was surprising, but not complaining. So basically I'll assume the code will work regardless of the mode used by the encryption method. Just thought I'd let yous know.

The guys here love Fortran as well. They're mathematicians and love it because they say formulas translate to Fortran nicely. My job is to create nice pretty front ends that wrap around their Fortran code. Unfortunately for the maths guys the overseers have decided everyone is moving over to C#.






Re: Visual Basic Express Edition Cryptography of packets!

Evoluator

Ok Guys, I have to declare I HATE System.Security.Cryptography class in .NET. I have maid my mind up. I am setting up a completely new Cryptography Algorithm.

Last night I came up with an amazing idea for cryptography, which the key size could have length of up to 2048 bits! I am not sure what to call it yet!

I will first sort out the mathematics behind it, then implement it in VB .NET, and share my results on this thread when It gets any where.






Re: Visual Basic Express Edition Cryptography of packets!

Derek Smyth

If you get it all working and it proves secure enough to be considered an industry standard then you can call it Dereks Encryption Algorithm (DEA), I don't mind.

Yeah, lets us know what you do with this. It would be good to see what you come up with.






Re: Visual Basic Express Edition Cryptography of packets!

ReneeC

I'll leave since this conversation is just for guys.

But I wanted to say this about fortan.

If you have two integers, and I almost never used floating point number since I so systems programming.....

A = 1

B =2

C = (A + B)/2

the answer will reliably be 1 because that's integer aritmetic.

in VB the answer will be 2 and that's horrible.