Changes between Version 6 and Version 7 of rfc47_dataset_caching

Timestamp:

Jul 29, 2014, 12:32:25 PM (10 years ago)

Author:

flippmoke

Comment:

--

rfc47_dataset_caching

@@ -93 +93 @@
 === Solution 1 (RW Mutex in GDALDataset ) ===
 
-For solution one the three mutexes are:
+==== Mutexes ====
+
+For solution 1 the three mutexes are:
 
  * Dataset RW Mutex (per GDALDataset)
@@ -101 +103 @@
 In order to prevent deadlocks, a priority of the mutexes is established in the order they are listed. For example if you have the Band Mutex, you may not obtain the Dataset RW Mutex, unless it was obtained prior to the Band Mutex being obtained. However, the goal should always be to never have more then mutex at a time!
 
-=== Dataset RW Mutex ===
+===== Dataset RW Mutex =====
 
 The objective of the Dataset RW Mutex is to protect the data stored within the the GDALRasterBlocks associated with a dataset, and lock during large Read or Write operations. This prevents two different threads from using memcpy on the same GDALRasterBlock at the same time. This mutex normally lies within the GDALDataset, but in the case of a standalone GDALRasterBand, it utilizes a new mutex on the Band. 
 
-=== Band Mutex ===
+===== Band Mutex =====
 
 The objective of the Band Mutex is to manage the control of the array of blocks in the GDALRasterBand, and manages the locking of the GDALRasterBlocks. This is a per GDALRasterBand Mutex.
 
-=== RBM Mutex ===
+===== RBM Mutex =====
 
 The objective of the RBM Mutex is to manage control of the LRU cache. This mutex is responsible for the control of the management of the cache's linked list and total amount of data stored in the cache.
 
+==== Pros ====
 
+This is a much more simple solution of the two different possible solutions. Since the protection of the Blocks are done at the Dataset level, it prevents the problem of some drivers such as geotiff where more then one band might be accessed in the reading or writing of one band. Therefore with out this protection here it could cause issues if locking was just at a band level per block's data.
 
+==== Cons ====
 
+This solution is not perhaps the most optimal way to lock because the protection of the IReadBlock, IWriteBlock, and IRasterIO routines is over the entire dataset. This is very limiting when you are reading the same dataset in a threaded environment, because it is not possible to read more then one block at a time. 
+
+=== Solution 2 (RW Mutex in GDALRasterBlock ) ===
+
+==== Mutexes ====
+
+For solution 2 the three mutexes are:
+
+ * Band Mutex (per GDALRasterBand)
+   * RBM Mutex (per GDALRasterBlockManager)
+   * Block RW Mutex (per GDALRasterBlock)
+
+In order to prevent deadlocks the band mutex has priority. This means that you can not get the Band Mutex if you have the RBM or Blow RW Mutex, unless you already had the Band Mutex prior to this. You may not obtain the Block mutex and the RBM mutex at the same time.
+
+===== Band Mutex =====
+
+The objective of the Band Mutex is to manage the control of the array of blocks in the GDALRasterBand, and manages the locking of the GDALRasterBlocks. This is a per GDALRasterBand Mutex.
+
+===== RBM Mutex =====
+
+The objective of the RBM Mutex is to manage control of the LRU cache. This mutex is responsible for the control of the management of the cache's linked list and total amount of data stored in the cache.
+
+===== Block RW Mutex =====
+
+The objective of the Block RW Mutex is to protect the data stored within the the GDALRasterBlocks associated with a dataset, and lock during large Read or Write operations. This prevents two different threads from using memcpy on the same GDALRasterBlock at the same time. It is created on a per block basis.
+
+==== Pros ====
+
+This is probably the most complete solution to making an intensive and fast threaded solution for the blocking. This is because the IReadWrite, IWriteBlock, and IRasterIO now are able to possibly pass a mutex with their calls, as a void pointer pointer. A change was made to the mutex as well such that a void pointer pointer that is NULL passed to CPLMutexHolderD, will not result in any pointer being created or any locking to occur. This means much of the behavior of the existing code can be maintained by simply passing a NULL value for the mutex. All of these changes allow the drivers to maintain much more control over the way that locking occurs when protecting the data inside a block. 
+
+==== Cons ====
+
+Obviously, this is a much more complex solution and therefore is harder to manage. It means that writing a driver is not as trivial as before and care must be taken in how locking is done within the driver in order to prevent deadlocks and maintain thread safety. The other issue that might arise from this is a slight slow down in non-threaded code because of the extra cycles spent locking data that will not be accessed in a threaded manner. Additionally, it might have issues in windows if too many mutexes are created (as there are quite a few more since it is a per GDALRasterBlock mutex). (Note: Not sure how I will be able to test this properly?)