QIODevice::read() and QFile::atEnd() performance improvements

atEnd improvements, since atEnd is used in published example Qt read
loops. The two main ideas are to push the buffer test forward, which
optimises the normal buffered case, and to cache the file size till you
get to the end, which otherwise has to get the file size every 16K of
data.

read buffer improvements: The ring buffer structure was causing
significant performance overheads. In practice QIODevice has much
simpler requirements on its read buffer, and a linear buffer can be
used instead. This now uses a buffer optimised to QIODevice's use of
it.

QIODevice read function improvements. There are a number of sub-themes
here, which all are aimed at getting the normal buffered path through
the code done as fast as possible. This gives greatest improvements for
small reads, but it is these small reads that have the biggest
problems.
- removing the isSequential test, by setting a pointer to the qint64
  to update and using a dummy qint64 target when isSequential, then
  writing through the pointer.
- doing the readability check on the first read only, out of the fast
  path
- doing the maxSize<0 test after the getchar fast path
- removing the buffer isEmpty test and giving a fast exit test
  instead
- moving arithmetic out of the fast path, so "data" and "maxSize"
  are now dynamically updating
- for RCVT builds, ARM mode is used for the read functions because
  the 64-bit operations are much slower with this compiler in Thumb
  mode

There are some other changes to read() which improve clarity:
- bytesToBuffer is now always set to QIODEVICE_BUFFERSIZE. This has
  always been the case, its just that the logic was not clear before.
- moreToRead is set to false now in the case where the request was
  met by a direct device read. Leaving it true in this case was
  pointless, and setting it true in the converse case seems dangerous
  because the function might iterate for a very long time, although
  it might meet the API semantics better and would be a change in
  behavior because the function used to not read more when it claimed
  it would.

Reviewed-by: Joao
Reviewed-by: Aleksandar Babic

1 files changed, 35 insertions, 11 deletions

diff --git a/src/corelib/io/qfile.cpp b/src/corelib/io/qfile.cpp
index aa1c7d9..50e9a8f 100644
--- a/src/corelib/io/qfile.cpp
+++ b/src/corelib/io/qfile.cpp
@@ -90,7 +90,8 @@ QFile::DecoderFn QFilePrivate::decoder = locale_decode;
 
 QFilePrivate::QFilePrivate()
     : fileEngine(0), lastWasWrite(false),
-      writeBuffer(QFILE_WRITEBUFFER_SIZE), error(QFile::NoError)
+      writeBuffer(QFILE_WRITEBUFFER_SIZE), error(QFile::NoError),
+      cachedSize(0)
 {
 }
 
@@ -1257,8 +1258,10 @@ QFile::resize(qint64 sz)
         seek(sz);
     if(d->fileEngine->setSize(sz)) {
         unsetError();
+        d->cachedSize = sz;
         return true;
     }
+    d->cachedSize = 0;
     d->setError(QFile::ResizeError, d->fileEngine->errorString());
     return false;
 }
@@ -1420,7 +1423,8 @@ qint64 QFile::size() const
     Q_D(const QFile);
     if (!d->ensureFlushed())
         return 0;
-    return fileEngine()->size();
+    d->cachedSize = fileEngine()->size();
+    return d->cachedSize;
 }
 
 /*!
@@ -1446,16 +1450,16 @@ bool QFile::atEnd() const
 {
     Q_D(const QFile);
 
+    // If there's buffered data left, we're not at the end.
+    if (!d->buffer.isEmpty())
+        return false;
+
     if (!isOpen())
         return true;
 
     if (!d->ensureFlushed())
         return false;
 
-    // If there's buffered data left, we're not at the end.
-    if (!d->buffer.isEmpty())
-        return false;
-
     // If the file engine knows best, say what it says.
     if (d->fileEngine->supportsExtension(QAbstractFileEngine::AtEndExtension)) {
         // Check if the file engine supports AtEndExtension, and if it does,
@@ -1463,6 +1467,11 @@ bool QFile::atEnd() const
         return d->fileEngine->atEnd();
     }
 
+    // if it looks like we are at the end, or if size is not cached,
+    // fall through to bytesAvailable() to make sure.
+    if (pos() < d->cachedSize)
+        return false;
+
     // Fall back to checking how much is available (will stat files).
     return bytesAvailable() == 0;
 }
@@ -1502,12 +1511,21 @@ qint64 QFile::readLineData(char *data, qint64 maxlen)
     if (!d->ensureFlushed())
         return -1;
 
-    if (d->fileEngine->supportsExtension(QAbstractFileEngine::FastReadLineExtension))
-        return d->fileEngine->readLine(data, maxlen);
+    qint64 read;
+    if (d->fileEngine->supportsExtension(QAbstractFileEngine::FastReadLineExtension)) {
+        read = d->fileEngine->readLine(data, maxlen);
+    } else {
+        // Fall back to QIODevice's readLine implementation if the engine
+        // cannot do it faster.
+        read = QIODevice::readLineData(data, maxlen);
+    }
+
+    if (read < maxlen) {
+        // failed to read all requested, may be at the end of file, stop caching size so that it's rechecked
+        d->cachedSize = 0;
+    }
 
-    // Fall back to QIODevice's readLine implementation if the engine
-    // cannot do it faster.
-    return QIODevice::readLineData(data, maxlen);
+    return read;
 }
 
 /*!
@@ -1528,6 +1546,12 @@ qint64 QFile::readData(char *data, qint64 len)
             err = QFile::ReadError;
         d->setError(err, d->fileEngine->errorString());
     }
+
+    if (read < len) {
+        // failed to read all requested, may be at the end of file, stop caching size so that it's rechecked
+        d->cachedSize = 0;
+    }
+
     return read;
 }