Make split_and_compare_prnus_of_subgroups.py compatible with RAW images #59

New Issue

Benjamin_Loison · 2024-04-27T14:34:14+02:00

Benjamin_Loison commented

2024-04-27 14:34:14 +02:00

>>> (executing file "split_and_compare_prnus_of_subgroups.py")
100%|██████████| 100/100 [55:15<00:00, 36.65s/it]

``` >>> (executing file "split_and_compare_prnus_of_subgroups.py") 100%|██████████| 100/100 [55:15<00:00, 36.65s/it] ``` ![need_to_implement_clever_mean_for_split_and_compare_prnus_of_subgroups.py](/attachments/cfab1e91-9330-4d01-92dc-3ad10309c2d0)

Benjamin_Loison added the

enhancement

medium priority

medium

labels 2024-04-27 14:35:24 +02:00

Benjamin_Loison pinned this 2024-04-27 15:01:43 +02:00

Benjamin_Loison commented

2024-04-27 15:38:13 +02:00

Trying to focus on 23/04/24 Rafael images, verifying that there are 253 as claimed.

3552 - 3294 + 1 = 259.

So a bit more than claimed but it is the same order of magnitude.

Let us check my notes and manually check if I notice differences. There is nothing to declare both in my notes and after having manually checked for this range of images.

for imageIndex in `seq 3294 3552`
do
    FILE=DSC0$imageIndex.ARW
    ln -s "/mnt/hdd0/robust_image_source_identification_on_modern_smartphones/rafael_240424/$FILE" "/home/benjamin/robust_image_source_identification_on_modern_smartphones/datasets/raise/rafael/230424/$FILE"
done

Estimated PRNU with wavelet denoiser:

Modified brightness and contrast:

We observe small circles and greater ones specific to the lens but as before (see issues/49#issuecomment-1693) lines artifacts but here also blocks, maybe due to the denoiser method, I am running other denoisers to investigate. Note that small circles match with those that we see on initial images.

Also note that we notice lens circle on the initial images, here DSC03552.JPG:

Estimated PRNU with mean denoiser:

Estimated PRNU with bilateral denoiser:

Modified contrast and brightness:

./extract_noise.py

Denoising images for color red: 100%|██████████████████████████████████████████| 259/259 [28:53:42<00:00, 401.63s/it]
Denoising images for color green_right: 100%|██████████████████████████████████| 259/259 [28:40:03<00:00, 398.47s/it]
Denoising images for color green_bottom: 100%|█████████████████████████████████| 259/259 [29:27:57<00:00, 409.57s/it]
Denoising images for color blue: 100%|█████████████████████████████████████████| 259/259 [29:17:09<00:00, 407.06s/it]

Trying to focus on 23/04/24 Rafael images, verifying that there are 253 as claimed. ![Figure_1](/attachments/341cc70d-645d-4631-80c2-59b1d549e616) 3552 - 3294 + 1 = 259. So a bit more than claimed but it is the same order of magnitude. Let us check my notes and manually check if I notice differences. There is nothing to declare both in my notes and after having manually checked for this range of images. ```bash for imageIndex in `seq 3294 3552` do FILE=DSC0$imageIndex.ARW ln -s "/mnt/hdd0/robust_image_source_identification_on_modern_smartphones/rafael_240424/$FILE" "/home/benjamin/robust_image_source_identification_on_modern_smartphones/datasets/raise/rafael/230424/$FILE" done ``` #### Estimated PRNU with wavelet denoiser: ![mean_rafael_230424_wavelet_multiple_colors](/attachments/0e3363b9-51e7-4b0a-b237-43a31ef3bb3a) Modified brightness and contrast: ![image](/attachments/846eb3f8-2535-4b87-8e12-ccceefdcbedf) We observe small circles and greater ones specific to the lens but as before (see [issues/49#issuecomment-1693](https://gitea.lemnoslife.com/Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones/issues/49#issuecomment-1693)) lines artifacts but here also blocks, maybe due to the denoiser method, I am running other denoisers to investigate. Note that small circles match with those that we see on initial images. Also note that we notice lens circle on the initial images, here `DSC03552.JPG`: ![DSC03552](/attachments/7cfe0da3-bce5-4c7a-9af8-e103159d20c9) ![image](/attachments/3523b891-9156-4084-be71-fece0d77c47d) ![image](/attachments/9e9ac906-0ccc-4111-8b06-5dbc716c11df) #### Estimated PRNU with mean denoiser: Similar results except no more blocks: ![mean_rafael_230424_mean_multiple_colors](/attachments/73552659-6d11-44de-937c-5714fc5aa3d8) Modified contrast and brightness: ![image](/attachments/7c429001-6648-42a9-87e4-11916fd31eaf) #### Estimated PRNU with bilateral denoiser: ![mean_rafael_230424_bilateral_multiple_colors](/attachments/6f5e7e36-f7d0-49b9-b2cd-362ab7948f72) Modified contrast and brightness: ![image](/attachments/7e0e5e8d-8cfa-4240-9f63-0930bc624a50) ```bash ./extract_noise.py ``` ``` Denoising images for color red: 100%|██████████████████████████████████████████| 259/259 [28:53:42<00:00, 401.63s/it] Denoising images for color green_right: 100%|██████████████████████████████████| 259/259 [28:40:03<00:00, 398.47s/it] Denoising images for color green_bottom: 100%|█████████████████████████████████| 259/259 [29:27:57<00:00, 409.57s/it] Denoising images for color blue: 100%|█████████████████████████████████████████| 259/259 [29:17:09<00:00, 407.06s/it] ```

Figure_1.svg

200 KiB

mean_rafael_230424_wavelet_multiple_colors.png

44 MiB

image.png

7.0 MiB

image.png

3.7 MiB

DSC03552.JPG

4.1 MiB

image.png

3.5 MiB

image.png

7.6 MiB

mean_rafael_230424_mean_multiple_colors.png

50 MiB

mean_rafael_230424_bilateral_multiple_colors.png

36 MiB

image.png

7.7 MiB

Benjamin_Loison commented

2024-04-27 16:44:10 +02:00

Avoid precising in b0fa01c6c4e2a6d0afb6cbd096e49c4a522e638d:

help(os.listdir)

Help on built-in function listdir in module posix:

listdir(path=None)
    Return a list containing the names of the files in the directory.
    
    path can be specified as either str, bytes, or a path-like object.  If path is bytes,
      the filenames returned will also be bytes; in all other circumstances
      the filenames returned will be str.
    If path is None, uses the path='.'.
    On some platforms, path may also be specified as an open file descriptor;\
      the file descriptor must refer to a directory.
      If this functionality is unavailable, using it raises NotImplementedError.
    
    The list is in arbitrary order.  It does not include the special
    entries '.' and '..' even if they are present in the directory.

The list is in arbitrary order.

At least the two consecutive executions returned an identical list.

Avoid precising in [b0fa01c6c4e2a6d0afb6cbd096e49c4a522e638d](https://gitea.lemnoslife.com/Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones/commit/b0fa01c6c4e2a6d0afb6cbd096e49c4a522e638d): ```python help(os.listdir) ``` ``` Help on built-in function listdir in module posix: listdir(path=None) Return a list containing the names of the files in the directory. path can be specified as either str, bytes, or a path-like object. If path is bytes, the filenames returned will also be bytes; in all other circumstances the filenames returned will be str. If path is None, uses the path='.'. On some platforms, path may also be specified as an open file descriptor;\ the file descriptor must refer to a directory. If this functionality is unavailable, using it raises NotImplementedError. The list is in arbitrary order. It does not include the special entries '.' and '..' even if they are present in the directory. ``` > The list is in arbitrary order. At least the two consecutive executions returned an identical list.

Benjamin_Loison referenced this issue from a commit

2024-04-27 16:49:10 +02:00

#59: Leverage `iterativeMean` to not suffer of RAM quantity issue

Benjamin_Loison commented

2024-04-27 17:55:17 +02:00

def tqdm(x, y = None):
    return x

shows that the program is Killed probably because of insufficient RAM.

I verified the usage of subgroupIterativeMean = subgroupsIterativeMean[subgroupIndex] with:

# check *pointer* mutation
print(subgroupIterativeMean.numberOfElementsInMean)

```py def tqdm(x, y = None): return x ``` shows that the program is `Killed` probably because of insufficient RAM. I verified the usage of `subgroupIterativeMean = subgroupsIterativeMean[subgroupIndex]` with: ```py # check *pointer* mutation print(subgroupIterativeMean.numberOfElementsInMean) ```

Benjamin_Loison commented

2024-04-27 18:52:26 +02:00

When will compare 2 devices estimated PRNUs the resolution will matter, could for instance consider only the minimal resolution of both devices. Note that corners may introduce a bias as for one device using the minimal resolution may contain all the corners and not for the other.

Concerning what is planned, that is RAISE flat-field and Rafael 23/04/24:

exiftool flat_001.NEF

Interesting fields found manually seem to be:

Image Width                     : 4992
Image Height                    : 3280
Rows Per Strip                  : 3280
Raw Image Center                : 2496 1640

exiftool DSC03552.ARW

Image Width                     : 6048
Image Height                    : 4024
Rows Per Strip                  : 4024
Sony Image Width                : 6000
Full Image Size                 : 6000x4000
Sony Image Height               : 4000
Exif Image Width                : 6000
Exif Image Height               : 4000

Should check actually with rawpy.

import rawpy

imageFilePath = 'rafael/230424/DSC03552.ARW'

with rawpy.imread(imageFilePath) as raw:
    rawImageVisibleShape = raw.raw_image_visible.copy().shape
    print(rawImageVisibleShape)

RAISE flat-field: (3280, 4948)
Rafael 23/04/24: (4024, 6024)

When will compare 2 devices estimated PRNUs the resolution will matter, could for instance consider only the minimal resolution of both devices. Note that corners may introduce a bias as for one device using the minimal resolution may contain all the corners and not for the other. Concerning what is planned, that is RAISE flat-field and Rafael 23/04/24: ```bash exiftool flat_001.NEF ``` Interesting fields found manually seem to be: ``` Image Width : 4992 Image Height : 3280 Rows Per Strip : 3280 Raw Image Center : 2496 1640 ``` ```bash exiftool DSC03552.ARW ``` ``` Image Width : 6048 Image Height : 4024 Rows Per Strip : 4024 Sony Image Width : 6000 Full Image Size : 6000x4000 Sony Image Height : 4000 Exif Image Width : 6000 Exif Image Height : 4000 ``` Should check actually with `rawpy`. ```py import rawpy imageFilePath = 'rafael/230424/DSC03552.ARW' with rawpy.imread(imageFilePath) as raw: rawImageVisibleShape = raw.raw_image_visible.copy().shape print(rawImageVisibleShape) ``` RAISE flat-field: `(3280, 4948)` Rafael 23/04/24: `(4024, 6024)`

Benjamin_Loison commented

2024-04-27 19:20:18 +02:00

RMS between estimated PRNU of 2 subgroups of raise flat-field TIF using wavelet denoiser:

PRNU subgroup 0:

Contrast and brightness modified:

PRNU subgroup 1:

So the blocking artifacts maybe comes from TIF usage.

RMS between estimated PRNU of 2 subgroups of raise flat-field TIF using wavelet denoiser: ![rms_between_estimated_prnu_of_2_subgroups_of_raise_flat_field_using_wavelet_denoiser](/attachments/52c1e48d-cfcc-4f7c-ab83-ef2e7d16ae70) PRNU subgroup 0: ![prnu_subgroup_0](/attachments/fbab5ea3-e97e-4c93-b2a2-d31188c3bae0) Contrast and brightness modified: ![image](/attachments/fea8cdd2-f8c5-4515-88ea-5c6a7d8accfa) PRNU subgroup 1: ![prnu_subgroup_1](/attachments/316c1e85-377d-47fc-9a44-89b1dffd8d24) So the blocking artifacts maybe comes from TIF usage.

rms_between_estimated_prnu_of_2_subgroups_of_raise_flat_field_using_wavelet_denoiser.svg

35 KiB

prnu_subgroup_0.png

38 MiB

prnu_subgroup_1.png

38 MiB

image.png

6.6 MiB

Benjamin_Loison commented

2024-04-27 20:46:30 +02:00

Forgot to add to 80a33428d6993bcd25c107bf4461d40df7d5244b:

import matplotlib.pyplot as plt

X = range(10)
plt.plot(X)
plt.show(block = False)

Forgot to add to [80a33428d6993bcd25c107bf4461d40df7d5244b](https://gitea.lemnoslife.com/Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones/commit/80a33428d6993bcd25c107bf4461d40df7d5244b): ```py import matplotlib.pyplot as plt X = range(10) plt.plot(X) plt.show(block = False) ```

Benjamin_Loison commented

2024-04-27 20:55:20 +02:00

Related to Benjamin_Loison/rawpy/issues/3.

Related to [Benjamin_Loison/rawpy/issues/3](https://codeberg.org/Benjamin_Loison/rawpy/issues/3).

Benjamin_Loison referenced this issue from a commit

2024-04-27 23:14:11 +02:00

#59: Should make `split_and_compare_prnus_of_subgroups.py` compatible with RAW images

Benjamin_Loison commented

2024-04-27 23:17:23 +02:00

Maybe mergeSingleColorChannelImagesAccordingToBayerFilter is not optimized.

Maybe [`mergeSingleColorChannelImagesAccordingToBayerFilter`](https://gitea.lemnoslife.com/Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones/src/commit/5647ab401923bac767b8e5040f3ea315a4eac1a9/datasets/raise/utils.py#L98) is not optimized.

Benjamin_Loison commented

2024-04-28 01:35:46 +02:00

RAISE flat-field raw images wavelet denoiser PRNU estimation among 2 subgroups:

RAISE flat-field raw images bilateral denoiser PRNU estimation among 2 subgroups:

Not the same y axis scale but seems smoother.

A bit surprising that do not have as good estimated PRNUs as expected.
In fact it seems that it is because I forgot the [0, 1] rescaling before denoising. Also suspicious that do not have low y-axis values as above.
Otherwise denoting on the x-axis the images considered could help, also for reproducibility and better comparison. Well in fact there are 2 subgroups with different images each, so it is not as clean as I initially had in mind.

RAISE flat-field raw images wavelet denoiser PRNU estimation among 2 subgroups: ![flat-field_nef_wavelet_rms_between_estimated_prnu_of_2_subgroups](/attachments/892a0716-e4e3-4ce6-804e-6e6ec7f77b7a) ![flat-field_nef_wavelet_estimated_prnu_subgroup_0](/attachments/9de517ef-729e-4e01-bf58-c055bd36d04f) ![flat-field_nef_wavelet_estimated_prnu_subgroup_1](/attachments/5b1acfcc-fa67-4fc1-a96e-654bdc441e0b) RAISE flat-field raw images bilateral denoiser PRNU estimation among 2 subgroups: ![flat-field_nef_bilateral_rms_between_estimated_prnu_of_2_subgroups](/attachments/0e87d5fe-78de-4546-8fd1-e9e895af8c8f) Not the same y axis scale but seems smoother. ![flat-field_nef_bilateral_estimated_prnu_subgroup_0](/attachments/d59b888f-9ae2-4cef-b96e-1db27fa5b098) ![flat-field_nef_bilateral_estimated_prnu_subgroup_1](/attachments/4797a043-26c4-45e1-8a82-3b47b2a3c7e9) A bit surprising that do not have as good estimated PRNUs as expected. In fact it seems that it is because I forgot the [0, 1] rescaling before denoising. Also suspicious that do not have low y-axis values [as above](https://gitea.lemnoslife.com/Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones/issues/59#issuecomment-1727). Otherwise denoting on the x-axis the images considered could help, also for reproducibility and better comparison. Well in fact there are 2 subgroups with different images each, so it is not as clean as I initially had in mind.

flat-field_nef_wavelet_estimated_prnu_subgroup_0.png

12 MiB

flat-field_nef_wavelet_estimated_prnu_subgroup_1.png

12 MiB

flat-field_nef_wavelet_rms_between_estimated_prnu_of_2_subgroups.svg

34 KiB

flat-field_nef_bilateral_rms_between_estimated_prnu_of_2_subgroups.svg

34 KiB

flat-field_nef_bilateral_estimated_prnu_subgroup_0.png

11 MiB

flat-field_nef_bilateral_estimated_prnu_subgroup_1.png

11 MiB

Benjamin_Loison commented

2024-04-29 15:40:34 +02:00

RAISE flat-field raw images bilateral right green denoiser PRNU estimation among 2 subgroups:

RAISE flat-field raw images bilateral right green denoiser PRNU estimation among 2 subgroups: ![flat-field_nef_bilateral_green_right_rms_between_estimated_prnu_of_2_subgroups](/attachments/123972f5-74b0-444f-bacd-4f51c80075c6) ![flat-field_nef_bilateral_green_right_estimated_prnu_subgroup_0](/attachments/bda93b09-2351-48cb-a28e-e3e0842f40e7) ![flat-field_nef_bilateral_green_right_estimated_prnu_subgroup_1](/attachments/f747a86d-58b7-4811-ad84-fb19388c1d82)

flat-field_nef_bilateral_green_right_rms_between_estimated_prnu_of_2_subgroups.svg

32 KiB

flat-field_nef_bilateral_green_right_estimated_prnu_subgroup_0.png

4.3 MiB

flat-field_nef_bilateral_green_right_estimated_prnu_subgroup_1.png

4.3 MiB

Benjamin_Loison referenced this issue from a commit

2024-04-30 01:04:46 +02:00

#59: Save `rmss` for futher investigations

Benjamin_Loison referenced this issue

2024-04-30 01:19:36 +02:00

Denoising time electricity consumption order of magnitude #60

Benjamin_Loison referenced this issue from a commit

2024-04-30 02:08:34 +02:00

Add and use `rescaleRawImageForDenoiser` and `updateExtremes` (#59)

Benjamin_Loison commented

2024-04-30 03:32:19 +02:00

With corrected RAW image denoiser:

RAISE flat-field raw images wavelet denoiser PRNU estimation among 2 subgroups:

Modified contrast and brightness:

RAISE flat-field raw images bilateral denoiser PRNU estimation among 2 subgroups:

With corrected RAW image denoiser: RAISE flat-field raw images wavelet denoiser PRNU estimation among 2 subgroups: ![flat-field_nef_wavelet_rms_between_estimated_prnu_of_2_subgroups](/attachments/5ac41f48-952e-47bc-8157-d284c701ef82) ![flat-field_nef_wavelet_estimated_prnu_subgroup_0](/attachments/be9f853b-0b80-460b-897a-6657a7878027) Modified contrast and brightness: ![image](/attachments/675d2bc2-8f1d-4703-b9ed-bb05362a1d23) ![flat-field_nef_wavelet_estimated_prnu_subgroup_1](/attachments/a493c921-d1d3-4341-ae2d-8920fc3d092d) RAISE flat-field raw images bilateral denoiser PRNU estimation among 2 subgroups: ![flat-field_nef_bilateral_rms_between_estimated_prnu_of_2_subgroups](/attachments/30f2325b-9b0a-48c7-bec5-b564df5b7320) ![flat-field_nef_bilateral_estimated_prnu_subgroup_0](/attachments/abb6456f-c79e-4c45-829c-f49dbd25ea14) ![flat-field_nef_bilateral_estimated_prnu_subgroup_1](/attachments/a9e976f1-1293-4e6a-a8df-d0aaf51cf204)

flat-field_nef_wavelet_rms_between_estimated_prnu_of_2_subgroups.svg

36 KiB

flat-field_nef_wavelet_estimated_prnu_subgroup_0.png

17 MiB

flat-field_nef_wavelet_estimated_prnu_subgroup_1.png

17 MiB

image.png

5.8 MiB

flat-field_nef_bilateral_rms_between_estimated_prnu_of_2_subgroups.svg

36 KiB

flat-field_nef_bilateral_estimated_prnu_subgroup_0.png

20 MiB