Context-Adaptive Interpolator (CAI) #9

New Issue

2024-03-19T21:19:53+01:00

Benjamin_Loison commented

2024-03-19 21:19:53 +01:00

https://web.archive.org/web/20240319200454/https://ipolcore.ipol.im/api/blobs/staticData/blobs/blob_directory/9/f/9f04e2005fddb9d5512e2f42a3b826b019755717.jpg

https://ipolcore.ipol.im/api/blobs/staticData/blobs/blob_directory/9/f/9f04e2005fddb9d5512e2f42a3b826b019755717.jpg

955582e0f8/algorithms/context-adaptive_interpolator.py

Related to #10.

After whole first neighbor part, do not see much:

But if GIMP > Colors > Brightness-Contrast..., then get with Brightness 50:

https://web.archive.org/web/20240319200454/https://ipolcore.ipol.im/api/blobs/staticData/blobs/blob_directory/9/f/9f04e2005fddb9d5512e2f42a3b826b019755717.jpg ![image](/attachments/05debce7-a5eb-4c31-933e-6f972be731f0) ![image](/attachments/f36fad4e-7322-412a-9d35-eaa65c997ad4) https://ipolcore.ipol.im/api/blobs/staticData/blobs/blob_directory/9/f/9f04e2005fddb9d5512e2f42a3b826b019755717.jpg https://gitea.lemnoslife.com/Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones/src/commit/955582e0f8bbd7932872e5950d6d2e8e65c61836/algorithms/context-adaptive_interpolator.py Related to #10. After whole first neighbor part, do not see much: ![tmpcn627e2o](/attachments/6a28dd6b-bc8b-483c-ae81-0187e072f87b) But if GIMP > `Colors` > `Brightness-Contrast...`, then get with `Brightness` `50`: ![image](/attachments/bbb7f635-5d3c-4f10-b35c-dc63e4282955)

image.png

12 MiB

tmpcn627e2o.PNG

3.1 MiB

image.png

12 MiB

9f04e2005fddb9d5512e2f42a3b826b019755717.jpg

4.4 MiB

Benjamin_Loison added the Context-Adaptive Interpolator label 2024-03-21 10:08:01 +01:00

Benjamin_Loison commented

2024-03-21 16:25:07 +01:00

Now that have possibly a first version, should try to compare with other models to see if it works.

Can work with a random part of the image to reduce complexity.

Otherwise can consider a single photo but all its parts as different images, as https://lesc.dinfo.unifi.it/VISION/dataset/ flat dataset is about 8.7 GB. Flat looks interesting as it seems to not have a strong scene, see #2 and #3.

Now that have possibly a first version, should try to compare with other models to see if it works. Can work with a random part of the image to reduce complexity. Otherwise can consider a single photo but all its parts as different images, as https://lesc.dinfo.unifi.it/VISION/dataset/ `flat` dataset is about 8.7 GB. Flat looks interesting as it seems to not have a strong scene, see #2 and #3.

Benjamin_Loison commented

2024-03-22 11:18:22 +01:00

Switching to numpy instead of PIL.Image starting work:

diff --git a/algorithms/context_adaptative_interpolator/context_adaptive_interpolator.py b/algorithms/context_adaptative_interpolator/conte
xt_adaptive_interpolator.py
index da680b1..69062d9 100644
--- a/algorithms/context_adaptative_interpolator/context_adaptive_interpolator.py
+++ b/algorithms/context_adaptative_interpolator/context_adaptive_interpolator.py
@@ -1,14 +1,13 @@
 # Based on https://web.archive.org/web/20231116015653/http://nrl.northumbria.ac.uk/id/eprint/29339/1/Paper_accepted.pdf IV. B..
 
-from PIL import Image
 from statistics import mean, median
 from wiener_filter import wienerFilter
+import numpy as np
 
 # Assume greyscale PIL image passed.
 # What about other color channels? See #11.
-def contextAdaptiveInterpolator(I, IImage):
-    rImage = Image.new('L', (IImage.size[0] - 2, IImage.size[1] - 2))
-    r = rImage.load()
+def contextAdaptiveInterpolator(I):
+    rImage = np.empty((I.shape[0] - 2, I.shape[1] - 2))
 
     # This threshold is debatable. See #13.
     THRESHOLD = 20
@@ -16,8 +15,8 @@ def contextAdaptiveInterpolator(I, IImage):
     print('before for loops')
     # Equation (10)
     # Accelerate computation. See #15.
-    for m in range(1, IImage.size[0] - 1):
-        for n in range(1, IImage.size[1] - 1):
+    for m in range(1, I.shape[0] - 1):
+        for n in range(1, I.shape[1] - 1):
                 e = I[m, n + 1]
                 se = I[m + 1, n + 1]
                 s = I[m + 1, n]
@@ -44,6 +43,7 @@ def contextAdaptiveInterpolator(I, IImage):
 
     # Why need to rotate the image? See #14.
     #rImage.rotate(-90).show()
+    exit(1)
 
     Q = 3
     # $\sigma_0^2$ is the noise variance.

<details> <summary>Switching to <code>numpy</code> instead of <code>PIL.Image</code> starting work:</summary> ```diff diff --git a/algorithms/context_adaptative_interpolator/context_adaptive_interpolator.py b/algorithms/context_adaptative_interpolator/conte xt_adaptive_interpolator.py index da680b1..69062d9 100644 --- a/algorithms/context_adaptative_interpolator/context_adaptive_interpolator.py +++ b/algorithms/context_adaptative_interpolator/context_adaptive_interpolator.py @@ -1,14 +1,13 @@ # Based on https://web.archive.org/web/20231116015653/http://nrl.northumbria.ac.uk/id/eprint/29339/1/Paper_accepted.pdf IV. B.. -from PIL import Image from statistics import mean, median from wiener_filter import wienerFilter +import numpy as np # Assume greyscale PIL image passed. # What about other color channels? See #11. -def contextAdaptiveInterpolator(I, IImage): - rImage = Image.new('L', (IImage.size[0] - 2, IImage.size[1] - 2)) - r = rImage.load() +def contextAdaptiveInterpolator(I): + rImage = np.empty((I.shape[0] - 2, I.shape[1] - 2)) # This threshold is debatable. See #13. THRESHOLD = 20 @@ -16,8 +15,8 @@ def contextAdaptiveInterpolator(I, IImage): print('before for loops') # Equation (10) # Accelerate computation. See #15. - for m in range(1, IImage.size[0] - 1): - for n in range(1, IImage.size[1] - 1): + for m in range(1, I.shape[0] - 1): + for n in range(1, I.shape[1] - 1): e = I[m, n + 1] se = I[m + 1, n + 1] s = I[m + 1, n] @@ -44,6 +43,7 @@ def contextAdaptiveInterpolator(I, IImage): # Why need to rotate the image? See #14. #rImage.rotate(-90).show() + exit(1) Q = 3 # $\sigma_0^2$ is the noise variance. ``` </details>

Benjamin_Loison commented

2024-03-22 11:58:00 +01:00

Output:

initialRmsDiff=39.478491226631874 caiRmsDiff=33.92120698086231
initialRmsDiff=40.08881131897964 caiRmsDiff=33.537350852224804
initialRmsDiff=40.289355638113335 caiRmsDiff=34.269085008282474
initialRmsDiff=40.55530379636706 caiRmsDiff=34.24951293546139
initialRmsDiff=40.24592977285094 caiRmsDiff=34.408401377102415
initialRmsDiff=41.08516426468623 caiRmsDiff=34.43268856222687
initialRmsDiff=40.92597941425012 caiRmsDiff=34.56995727347227
initialRmsDiff=40.07153637363746 caiRmsDiff=33.87700023084982
initialRmsDiff=40.78248621210611 caiRmsDiff=34.21882434634775
initialRmsDiff=41.118697869134 caiRmsDiff=35.11659633548482

shows interesting results but is it enough? Let us generate multiple PRNUs and see if it classifies correctly thanks to rmsdiff.

<details> <summary>Output:</summary> ``` initialRmsDiff=39.478491226631874 caiRmsDiff=33.92120698086231 initialRmsDiff=40.08881131897964 caiRmsDiff=33.537350852224804 initialRmsDiff=40.289355638113335 caiRmsDiff=34.269085008282474 initialRmsDiff=40.55530379636706 caiRmsDiff=34.24951293546139 initialRmsDiff=40.24592977285094 caiRmsDiff=34.408401377102415 initialRmsDiff=41.08516426468623 caiRmsDiff=34.43268856222687 initialRmsDiff=40.92597941425012 caiRmsDiff=34.56995727347227 initialRmsDiff=40.07153637363746 caiRmsDiff=33.87700023084982 initialRmsDiff=40.78248621210611 caiRmsDiff=34.21882434634775 initialRmsDiff=41.118697869134 caiRmsDiff=35.11659633548482 ``` </details> shows interesting results but is it enough? Let us generate multiple PRNUs and see if it classifies correctly thanks to `rmsdiff`.

Benjamin_Loison commented

2024-03-22 12:10:30 +01:00

RMS diff with average image = 141.44015650123464
RMS diff with average CAI images = 103.2666640411378

seems to show promising results.

``` RMS diff with average image = 141.44015650123464 RMS diff with average CAI images = 103.2666640411378 ``` seems to show promising results.

Benjamin_Loison commented

2024-03-22 12:45:51 +01:00

While being nearest to actual PRNU thanks to CAI, it does not seem to help:

RMS diff with average image = 100.26988483431852
RMS diff with average CAI images = 79.12339215595037
RMS diff with average image = 100.4561617308142
RMS diff with average CAI images = 79.25003405691945
RMS diff with average image = 100.41585407693348
RMS diff with average CAI images = 79.19415609779311
RMS diff with average image = 100.2946769115733
RMS diff with average CAI images = 79.22415971021594
RMS diff with average image = 100.15874850627752
RMS diff with average CAI images = 79.06999592755365
RMS diff with average image = 100.05679417484289
RMS diff with average CAI images = 79.01107846556306
RMS diff with average image = 100.14450374763148
RMS diff with average CAI images = 78.91739843174084
RMS diff with average image = 100.05990271262198
RMS diff with average CAI images = 78.99680738816862
RMS diff with average image = 100.43265900164461
RMS diff with average CAI images = 79.25623627547293
RMS diff with average image = 100.17477280943478
RMS diff with average CAI images = 79.07313788028885
correctGuessesByAverageImages / NUMBER_OF_PHONES=0.1
correctGuessesByAverageCAIImages / NUMBER_OF_PHONES=0.1

<details> <summary>While being nearest to actual PRNU thanks to CAI, it does not seem to help:</summary> ``` RMS diff with average image = 100.26988483431852 RMS diff with average CAI images = 79.12339215595037 RMS diff with average image = 100.4561617308142 RMS diff with average CAI images = 79.25003405691945 RMS diff with average image = 100.41585407693348 RMS diff with average CAI images = 79.19415609779311 RMS diff with average image = 100.2946769115733 RMS diff with average CAI images = 79.22415971021594 RMS diff with average image = 100.15874850627752 RMS diff with average CAI images = 79.06999592755365 RMS diff with average image = 100.05679417484289 RMS diff with average CAI images = 79.01107846556306 RMS diff with average image = 100.14450374763148 RMS diff with average CAI images = 78.91739843174084 RMS diff with average image = 100.05990271262198 RMS diff with average CAI images = 78.99680738816862 RMS diff with average image = 100.43265900164461 RMS diff with average CAI images = 79.25623627547293 RMS diff with average image = 100.17477280943478 RMS diff with average CAI images = 79.07313788028885 correctGuessesByAverageImages / NUMBER_OF_PHONES=0.1 correctGuessesByAverageCAIImages / NUMBER_OF_PHONES=0.1 ``` </details>

Benjamin_Loison commented

2024-03-22 12:51:50 +01:00

Output:

RMS diff with average image = 146.59313795571444
RMS diff with average CAI images = 145.5368224929786
Actual phone index 0, guessed phone index 7 by averages images and 9 by average CAI images
RMS diff with average image = 146.64370917774227
RMS diff with average CAI images = 143.13530906576742
Actual phone index 1, guessed phone index 7 by averages images and 7 by average CAI images
RMS diff with average image = 145.17654342133108
RMS diff with average CAI images = 143.6744109727487
Actual phone index 2, guessed phone index 9 by averages images and 9 by average CAI images
RMS diff with average image = 145.36891629977805
RMS diff with average CAI images = 145.55125576273915
Actual phone index 3, guessed phone index 7 by averages images and 4 by average CAI images
RMS diff with average image = 146.46235667852721
RMS diff with average CAI images = 146.01008029717366
Actual phone index 4, guessed phone index 9 by averages images and 9 by average CAI images
RMS diff with average image = 146.14840555172847
RMS diff with average CAI images = 143.08920851025619
Actual phone index 5, guessed phone index 3 by averages images and 9 by average CAI images
RMS diff with average image = 146.9506852867655
RMS diff with average CAI images = 144.3420511546433
Actual phone index 6, guessed phone index 7 by averages images and 9 by average CAI images
RMS diff with average image = 145.98354048246972
RMS diff with average CAI images = 144.3882787002275
Actual phone index 7, guessed phone index 9 by averages images and 9 by average CAI images
RMS diff with average image = 146.36162440221906
RMS diff with average CAI images = 142.80918864009286
Actual phone index 8, guessed phone index 7 by averages images and 4 by average CAI images
RMS diff with average image = 147.1752482958242
RMS diff with average CAI images = 143.80136100050768
Actual phone index 9, guessed phone index 7 by averages images and 9 by average CAI images
correctGuessesByAverageImages / NUMBER_OF_PHONES=0.0
correctGuessesByAverageCAIImages / NUMBER_OF_PHONES=0.1

<details> <summary>Output:</summary> ``` RMS diff with average image = 146.59313795571444 RMS diff with average CAI images = 145.5368224929786 Actual phone index 0, guessed phone index 7 by averages images and 9 by average CAI images RMS diff with average image = 146.64370917774227 RMS diff with average CAI images = 143.13530906576742 Actual phone index 1, guessed phone index 7 by averages images and 7 by average CAI images RMS diff with average image = 145.17654342133108 RMS diff with average CAI images = 143.6744109727487 Actual phone index 2, guessed phone index 9 by averages images and 9 by average CAI images RMS diff with average image = 145.36891629977805 RMS diff with average CAI images = 145.55125576273915 Actual phone index 3, guessed phone index 7 by averages images and 4 by average CAI images RMS diff with average image = 146.46235667852721 RMS diff with average CAI images = 146.01008029717366 Actual phone index 4, guessed phone index 9 by averages images and 9 by average CAI images RMS diff with average image = 146.14840555172847 RMS diff with average CAI images = 143.08920851025619 Actual phone index 5, guessed phone index 3 by averages images and 9 by average CAI images RMS diff with average image = 146.9506852867655 RMS diff with average CAI images = 144.3420511546433 Actual phone index 6, guessed phone index 7 by averages images and 9 by average CAI images RMS diff with average image = 145.98354048246972 RMS diff with average CAI images = 144.3882787002275 Actual phone index 7, guessed phone index 9 by averages images and 9 by average CAI images RMS diff with average image = 146.36162440221906 RMS diff with average CAI images = 142.80918864009286 Actual phone index 8, guessed phone index 7 by averages images and 4 by average CAI images RMS diff with average image = 147.1752482958242 RMS diff with average CAI images = 143.80136100050768 Actual phone index 9, guessed phone index 7 by averages images and 9 by average CAI images correctGuessesByAverageImages / NUMBER_OF_PHONES=0.0 correctGuessesByAverageCAIImages / NUMBER_OF_PHONES=0.1 ``` </details>

Benjamin_Loison commented

2024-03-22 12:53:48 +01:00