Changes between Version 7 and Version 8 of GSoC/2021/RasterParallelization

Timestamp:

Aug 2, 2021, 7:09:22 AM (3 years ago)

Author:

aaronsms

Comment:

--

GSoC/2021/RasterParallelization

@@ -293 +293 @@
 No, it has been good so far.
 
+=== Week 5 ===
+
+'''1) What did I get done this week?'''
+
+To benchmark both r.mfilter and r.neighbor implementation, I have made use of the recently merged benchmark library on randomly generated raster using r.surf.fractal.
+
+The preliminary result is as follows for both modules (y-axis - time/secs, x-axis - nprocs | benchmarked on my local workstation):
+Furthermore, checks are done to compare between performance on master branch vs after implementation (nprocs = 1), and the results are comparable.
+
+These two implementations make use of extensive disk I/O to write to temporary file buffer before transferring to the final raster file format. This behavior is default in r.mfilter, but is explicitly introduced in r.neighbors to allow for parallelization. Upon discussion with the mentors, we decided that we should make better use of memory over disk. Ideally, the user will be able to input the size of memory usage to be used for buffer. However, r.mfilter will still preserve its original usage of temporary files buffer.
+
+'''2) What do I plan on doing next week?'''
+
+    - Complete rework of r.neighbors implementation
+    - Compare benchmark between the two implementations
+
+'''3) Am I blocked on anything?'''
+
+No major roadblock, but I need to catch up a bit to rework my r.neighbor implementation.
+
+=== Week 6 ===
+
+'''1) What did I get done this week?'''
+
+r.neighbors
+
+The main goal that I have accomplished is to do a complete rework of the r.neighbors implementation (PR: [https://github.com/OSGeo/grass/pull/1724]). A benchmark script is ready under 'benchmark' directory for users to test the performance on their local machine. The performance is comparable to the previous implementation that make use of temporary files as buffer (on SSD) instead of memory. The result of the benchmarking on my local machine (12 cores) under the PR.
+
+r.mfilter
+
+There are issues pointed out when working on raster files > 2GB (PR: [https://github.com/OSGeo/grass/pull/1708]). This is promptly addressed with commit (4caa96), and the cause is due to overflow from multiplication. This PR is ready, and a benchmark script is provided as well for local benchmarking.
+
+'''2) What do I plan on doing next week?'''
+
+    - Introduce an environment variable that overwrites the default nprocs parameter which is currently 1. This is so that the users do not need to add nprocs parameter explicitly.
+    - Implement r.resamp.filter/r.resamp.interp parallelization
+
+'''3) Am I blocked on anything?'''
+No major issues.
+
+=== Week 7 ===
+
+'''1) What did I get done this week?'''
+
+- Introduce an environment variable that overwrites the default nprocs parameter which is currently 1. This is so that the users do not need to add nprocs parameter explicitly.
+
+r.resamp.filter
+
+    - Implement parallelization
+    - Add test cases
+
+'''2) What do I plan on doing next week?'''
+
+    - Implement parallelization for r.slope.aspect with testing and benchmarking
+
+'''3) Am I blocked on anything?'''
+
+No major issues.
+
+=== Week 8 ===
+
+'''1) What did I get done this week?'''
+
+r.resamp.interp [https://github.com/OSGeo/grass/pull/1771]
+
+- Implement parallelization
+
+r.slope.aspect [https://github.com/OSGeo/grass/pull/1767]
+
+- Implement parallelization
+
+Both implementation above follows similarly to r.neighbor [https://github.com/OSGeo/grass/pull/1724]. r.slope.aspect keeps track of global statistics variable like min/max, thus additional variable reduction is required aside from map computation. The benchmarking of the modules will be supplemented in the PR.
+
+'''2) What do I plan on doing next week?'''
+
+    - Refactor r.univar
+    - Implement parallelization for r.series, r.patch
+    - Revisit r.proj to decide on implementation
+
+'''3) Am I blocked on anything?'''
+
+No major issues.
+
 
 == Final report ==