How to do Mass Base64 Encoding In C# & .NET
[C#, .NET]
In a previous post, “How to Encode Base64 Data In C# & .NET”, we looked at various ways to encode data using the Base64 protocol.
This generally works fine when you have small amounts of data to encode, or the encoding is not in a critical path in your application.
When you have large amounts of data, or you want to encode data that might not fit in memory at once, you need a different approach - you need to use streams, and in particular, the CryptoStream. We then use the ToBase64Transform class to perform the encoding.
The code is as follows:
using System.Security.Cryptography;
using System.Text;
byte[] rawData = Encoding.UTF8.GetBytes("This is a dummy representing a large amount of text to Base64 encode. Quite a lot of data");
using (var inputStream = new MemoryStream(rawData))
{
using (var outputStream = new MemoryStream())
{
using (var transform = new ToBase64Transform())
{
using (var cryptoStream = new CryptoStream(outputStream, transform, CryptoStreamMode.Write, leaveOpen: true))
{
inputStream.CopyTo(cryptoStream);
// Important to flush the last bytes and add padding
cryptoStream.FlushFinalBlock();
}
}
// Fetch our data from the output stream
var base64Result = Encoding.ASCII.GetString(outputStream.ToArray());
Console.WriteLine(base64Result); //
}
}
If the data was in a file, and we want the encoded data in a second file, we can rewrite the code like this:
using (var inputStream = File.OpenRead("VeryLargeFileForEncoding.txt"))
{
using (var outputStream = File.Create("EncodedFile"))
{
using (var transform = new ToBase64Transform())
{
using (var cryptoStream = new CryptoStream(outputStream, transform, CryptoStreamMode.Write))
{
inputStream.CopyTo(cryptoStream);
cryptoStream.FlushFinalBlock();
}
}
}
}
This is much more memory efficient, and removes the need for intermediate buffers for processing.
If the data we want to encode is in a file, and we want the encoded string available immediately, this would be the way to go:
var rawData = File.ReadAllBytes("SmallOrMediumFile.txt");
var base64Result = Convert.ToBase64String(rawData);
Console.WriteLine(base64Result);
TLDR
There are Stream based classes that can help encoding large amounts of data.
The code is in my GitHub.
Happy hacking!