Faster FileStream with TBufferedFileStream

Buffering the File Stream in Delphi / C++ Builder

Part of the very fast FireDAC database component library is TFDFileStream,a class that allows high performance access to trace files, (TFDMoniFlatFileClientLink), text data file reading (TFDBatchMoveTextReader / TFDBatchMoveTextWrtiter), SQL Script file reading (TFDScript) and data serialization to file (TFDMemTable etc). Well, it seems it was too good to keep hidden and with the release of 10.1 Berlin, this has been moved from  FireDAC.Stan.Util to System.Classes and has been renamed to TBufferedFileStream.

TBufferedFileStream is a TFileStream descendant that optimises multiple consecutive small writes or reads. In other words, TBufferedFileStream adds buffering support to TFileStream.

As TBufferedFileStream descends from TFileStream it is a simple replacement to add it into your applications and gain the speed benefits it brings.

Read more for code example..

TBufferedFileStream Example

To see the speed difference, lets run a test! Firstly, using the old style of Read/Write using TFileStream and secondly compared to TBufferedFileStream.

Setting up the TBufferedFileStream Test.

This test uses a blank VCL forms application, (you could use a FireMonkey one instead); added to the main form is a TMemo (memo1) and 3 x TButton (named btnWrite, btnRead, btnReadBuffered. Additionally added to the uses is System.Diagnostics  to provide the TStopWatch for timing the speed of TFileStream and TBufferedFileStream.

Writing the file to test the reading methods is pretty simple. For the first button (btnWrite) here is some implemented code to quickly setup 100,000 rows into a text file. (you could make this bigger still if you wish)

procedure TFormReaderWriter.btnWriteClick(Sender: TObject);
var
  sw: TStreamWriter;
  I: Integer;
begin
  sw := TStreamWriter.Create('test.txt', False, TEncoding.UTF8);
  try
    // write 10K lines
    sw.WriteLine ('Hello, world');
    for I := 1 to 99999 do
      sw.WriteLine ('Hello ' + I.ToString);
 finally
   sw.Free;
 end;
 Memo1.Lines.Add ('File written');
end;

WriteLn in the code above adds the text and appends character 13 to the end of every line. (in fact you can go open it in notepad if you want to see a long list of Hello x) The following two tests will pass the file and count the number of times char 13 appears to get a total of the rows found.  This is a really crude test that is very incremental in the reading operation.

procedure TFormReaderWriter.btnReadClick(Sender: TObject);
var
 fStr: TFileStream;
 Total, I: Integer;
 sw: TStopwatch;
 ch: Char;
begin
  sw := TStopwatch.StartNew;
  fStr := TFileStream.Create('test.txt', fmOpenRead);
  try
    Total := 0;
    while fStr.Read (ch, 1) = 1 do begin
      if ch = #13 then
        Inc(Total);
    end;
    Memo1.Lines.Add ('Lines: ' + Total.ToString);
  finally
    fStr.Free;
  end;
  sw.Stop;
  Memo1.Lines.Add ('msec: ' + sw.ElapsedMilliseconds.ToString);
end;

Running this code inside my VM the StopWatch returns back in around 1573ms to read the 100,000 lines.

Now I know the above code may be simple to parse, but is not very efficient. (unlike loading a text file into a TStringList). So lets make the code as effective as the highly optimised TStringList (even though the code is not as efficient as it could be) by simply copying the above procedure into the 3rd button OnClick event and changing TFileStream for a TBufferedFileStream.

procedure TFormReaderWriter.btnReadBufferedClick(Sender: TObject);
var
  fStr: TBufferedFileStream;
  Total, I: Integer;
  sw: TStopwatch;
  ch: Char;
begin
  sw := TStopwatch.StartNew;
  fStr := TBufferedFileStream.Create('test.txt', fmOpenRead);
  try
    Total := 0;
    while fStr.Read (ch, 1) = 1 do begin
      if ch = #13 then
        Inc(Total);
    end;
    Memo1.Lines.Add ('Lines: ' + Total.ToString);
  finally
    fStr.Free;
  end;
  sw.Stop;
  Memo1.Lines.Add ('msec: ' +   sw.ElapsedMilliseconds.ToString);
end;

Running the updated code dramatically reduces the StopWatch time to just 12ms! Awesome!

4 thoughts on “Faster FileStream with TBufferedFileStream”

  1. A buffered file stream is very nice, but the part I don’t get is how how this was ever relevant to database streaming, which typically comes over a network, not from the file system…

Leave a Reply

Your email address will not be published. Required fields are marked *