Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones
1
0
Fork 0
Code Issues 68 Pull Requests Packages Projects Releases Wiki Activity

RAW image processing #49

New Issue
Open
opened 2024-04-15 11:34:15 +02:00 by Benjamin_Loison · 23 comments
Benjamin_Loison commented 2024-04-15 11:34:15 +02:00
Owner
Copy Link
import rawpy

help(rawpy)

help(rawpy)
Help on package rawpy:

NAME
    rawpy

PACKAGE CONTENTS
    _rawpy
    _version
    enhance

FUNCTIONS
    imread(pathOrFile)
        Convenience function that creates a :class:`rawpy.RawPy` instance, opens the given file,
        and returns the :class:`rawpy.RawPy` instance for further processing.
        
        :param str|file pathOrFile: path or file object of RAW image that will be read
        :rtype: :class:`rawpy.RawPy`

DATA
    absolute_import = _Feature((2, 5, 0, 'alpha', 1), (3, 0, 0, 'alpha', 0...
    flags = {'6BY9RPI': True, 'DEMOSAIC_PACK_GPL2': False, 'DEMOSAIC_PACK_...
    libraw_version = (0, 21, 1)

VERSION
    0.19.1

FILE
    /home/benjamin/.local/lib/python3.10/site-packages/rawpy/__init__.py

filename = 'flat_001.NEF'

raw = rawpy.imread(filename)
help(raw)

...
class RawPy(builtins.object)
 |  Load RAW images, work on their data, and create a postprocessed (demosaiced) image.
...
 |  close(self)
...
 |  postprocess(self, params=None, **kw)
...
 |  raw_color(self, row, column)
...
 |  raw_value(self, row, column)
...
 |  raw_value_visible(self, row, column)
...
 |  color_desc
...
 |  color_matrix
...
 |  num_colors
...
 |  raw_colors
...
 |  raw_image
...
 |  raw_pattern
...
 |  raw_type
...
 |  rgb_xyz_matrix
...
 |  sizes

skipping low-level methods proposing an alternative.

Do not get _visible meaning.

import rawpy

raw = rawpy.imread(filename)

data = raw.raw_image_visible.astype(np.float64)
δy, δx = np.argwhere(raw.raw_colors == 0)[0]  # offset on the raw
# red channel corresponds to data[δy::2, δx::2]
# blue channel corresponds to data[1 - δy::2, 1 - δx::2]
# first green channel corresponds to data[δy::2, 1 - δx::2]
# second green channel corresponds to data[1 - δy::2, δx::2]

import imageio

with rawpy.imread(filename) as raw:
    rgb = raw.postprocess()

imageio.imsave('default.tiff', rgb)

Note that according to the Stack Overflow answer 51304155 _visible array versions are not contiguous, hence imply very significant performance decrease. It seems that OWNDATA flag is also involved:

>>> raw.raw_colors_visible.flags
  C_CONTIGUOUS : False
  F_CONTIGUOUS : False
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False


>>> raw.raw_colors_visible.copy().flags
  C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
```py import rawpy help(rawpy) ``` ``` help(rawpy) Help on package rawpy: NAME rawpy PACKAGE CONTENTS _rawpy _version enhance FUNCTIONS imread(pathOrFile) Convenience function that creates a :class:`rawpy.RawPy` instance, opens the given file, and returns the :class:`rawpy.RawPy` instance for further processing. :param str|file pathOrFile: path or file object of RAW image that will be read :rtype: :class:`rawpy.RawPy` DATA absolute_import = _Feature((2, 5, 0, 'alpha', 1), (3, 0, 0, 'alpha', 0... flags = {'6BY9RPI': True, 'DEMOSAIC_PACK_GPL2': False, 'DEMOSAIC_PACK_... libraw_version = (0, 21, 1) VERSION 0.19.1 FILE /home/benjamin/.local/lib/python3.10/site-packages/rawpy/__init__.py ``` ```py filename = 'flat_001.NEF' raw = rawpy.imread(filename) help(raw) ``` ``` ... class RawPy(builtins.object) | Load RAW images, work on their data, and create a postprocessed (demosaiced) image. ... | close(self) ... | postprocess(self, params=None, **kw) ... | raw_color(self, row, column) ... | raw_value(self, row, column) ... | raw_value_visible(self, row, column) ... | color_desc ... | color_matrix ... | num_colors ... | raw_colors ... | raw_image ... | raw_pattern ... | raw_type ... | rgb_xyz_matrix ... | sizes ``` skipping *low-level methods* proposing an alternative. Do not get `_visible` meaning. ```py import rawpy raw = rawpy.imread(filename) data = raw.raw_image_visible.astype(np.float64) δy, δx = np.argwhere(raw.raw_colors == 0)[0] # offset on the raw # red channel corresponds to data[δy::2, δx::2] # blue channel corresponds to data[1 - δy::2, 1 - δx::2] # first green channel corresponds to data[δy::2, 1 - δx::2] # second green channel corresponds to data[1 - δy::2, δx::2] ``` ```py import imageio with rawpy.imread(filename) as raw: rgb = raw.postprocess() imageio.imsave('default.tiff', rgb) ``` Note that according to [the Stack Overflow answer 51304155](https://stackoverflow.com/a/51304155) `_visible` array versions are not contiguous, hence imply very significant performance decrease. It seems that `OWNDATA` flag is also involved: ```py >>> raw.raw_colors_visible.flags C_CONTIGUOUS : False F_CONTIGUOUS : False OWNDATA : False WRITEABLE : True ALIGNED : True WRITEBACKIFCOPY : False >>> raw.raw_colors_visible.copy().flags C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True WRITEBACKIFCOPY : False ```
Benjamin_Loison added the
enhancement
medium priority
medium
 labels 2024-04-15 11:34:15 +02:00
Benjamin_Loison commented 2024-04-15 12:08:10 +02:00
Author
Owner
Copy Link
def printWithName(variable):
    print(variable, '=', repr(eval(variable)))

with rawpy.imread(filename) as raw:
    printWithName('raw.color_desc')
    printWithName('raw.color_matrix')
    printWithName('raw.num_colors')
    print()
    
    printWithName('raw.raw_colors.shape')
    printWithName('raw.raw_colors')
    print()
    
    printWithName('raw.raw_colors_visible.shape')
    printWithName('raw.raw_colors_visible')
    print()
    
    printWithName('raw.raw_image_visible.shape')
    printWithName('raw.raw_image_visible.dtype')
    printWithName('raw.raw_image_visible')
    print()
    
    printWithName('raw.raw_image.min()')
    printWithName('raw.raw_image.max()')
    printWithName('raw.raw_image.shape')
    printWithName('raw.raw_image')
    print()
    
    printWithName('raw.raw_pattern')
    printWithName('raw.raw_type')
    printWithName('raw.rgb_xyz_matrix')
    printWithName('raw.sizes')
    print()
    
    δy, δx = np.argwhere(raw.raw_colors == 0)[0]
    printWithName('δy')
    printWithName('δx')

raw.color_desc = b'RGBG'
raw.color_matrix = array([[0., 0., 0., 0.],
       [0., 0., 0., 0.],
       [0., 0., 0., 0.]], dtype=float32)
raw.num_colors = 3

raw.raw_colors.shape = (3280, 4992)
raw.raw_colors = array([[0, 1, 0, ..., 1, 0, 1],
       [3, 2, 3, ..., 2, 3, 2],
       [0, 1, 0, ..., 1, 0, 1],
       ...,
       [3, 2, 3, ..., 2, 3, 2],
       [0, 1, 0, ..., 1, 0, 1],
       [3, 2, 3, ..., 2, 3, 2]], dtype=uint8)

raw.raw_colors_visible.shape = (3280, 4948)
raw.raw_colors_visible = array([[0, 1, 0, ..., 1, 0, 1],
       [3, 2, 3, ..., 2, 3, 2],
       [0, 1, 0, ..., 1, 0, 1],
       ...,
       [3, 2, 3, ..., 2, 3, 2],
       [0, 1, 0, ..., 1, 0, 1],
       [3, 2, 3, ..., 2, 3, 2]], dtype=uint8)

raw.raw_image_visible.shape = (3280, 4948)
raw.raw_image_visible.dtype = dtype('uint16')
raw.raw_image_visible = array([[ 864,  930,  854, ...,  647,  640,  658],
       [1062,  308,  977, ...,  219,  746,  219],
       [ 802,  919,  867, ...,  710,  625,  685],
       ...,
       [ 815,  322,  833, ...,  146,  511,  192],
       [ 769,  726,  751, ...,  513,  466,  468],
       [ 771,  321,  813, ...,  149,  525,  189]], dtype=uint16)

raw.raw_image.min() = 0
raw.raw_image.max() = 4650
raw.raw_image.shape = (3280, 4992)
raw.raw_image = array([[ 864,  930,  854, ...,    0,    0,    0],
       [1062,  308,  977, ...,    0,    0,    0],
       [ 802,  919,  867, ...,    0,    0,    0],
       ...,
       [ 815,  322,  833, ...,    0,    0,    0],
       [ 769,  726,  751, ...,    0,    0,    0],
       [ 771,  321,  813, ...,    0,    0,    0]], dtype=uint16)

raw.raw_pattern = array([[0, 1],
       [3, 2]], dtype=uint8)
raw.raw_type = <RawType.Flat: 0>
raw.rgb_xyz_matrix = array([[ 0.8198, -0.2239, -0.0724],
       [-0.4871,  1.2389,  0.2798],
       [-0.1043,  0.205 ,  0.7181],
       [ 0.    ,  0.    ,  0.    ]], dtype=float32)
raw.sizes = ImageSizes(raw_height=3280, raw_width=4992, height=3280, width=4948, top_margin=0, left_margin=0, iheight=3280, iwidth=4948, pixel_aspect=1.0, flip=0)

δy = 0
δx = 0

Why different raw_width and width?

raw_image seems to have 0 in this case at the right side while it does not in raw_image_visible.

Could verify color correctness in the RAW array. As in internship offer display image without interpolation would ease doing so it seems, in addition to be interesting.

help(rawpy.RawPy.postprocess)
help(rawpy.Params)
help(rawpy.DemosaicAlgorithm)

seem to show that we have to do it on our own.

How to compute PRNU on these arrays? Only considering a given color channel seems to make sense. Pay attention that green might have a different shape. More precisely it does not seem to have the grid shape as the 2 other channels.

(4992 - 4948) / 2

22.0

import math

math.gcd(3280, 4992)
math.gcd(3280, 4948)

16
4
```py def printWithName(variable): print(variable, '=', repr(eval(variable))) with rawpy.imread(filename) as raw: printWithName('raw.color_desc') printWithName('raw.color_matrix') printWithName('raw.num_colors') print() printWithName('raw.raw_colors.shape') printWithName('raw.raw_colors') print() printWithName('raw.raw_colors_visible.shape') printWithName('raw.raw_colors_visible') print() printWithName('raw.raw_image_visible.shape') printWithName('raw.raw_image_visible.dtype') printWithName('raw.raw_image_visible') print() printWithName('raw.raw_image.min()') printWithName('raw.raw_image.max()') printWithName('raw.raw_image.shape') printWithName('raw.raw_image') print() printWithName('raw.raw_pattern') printWithName('raw.raw_type') printWithName('raw.rgb_xyz_matrix') printWithName('raw.sizes') print() δy, δx = np.argwhere(raw.raw_colors == 0)[0] printWithName('δy') printWithName('δx') ``` ``` raw.color_desc = b'RGBG' raw.color_matrix = array([[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]], dtype=float32) raw.num_colors = 3 raw.raw_colors.shape = (3280, 4992) raw.raw_colors = array([[0, 1, 0, ..., 1, 0, 1], [3, 2, 3, ..., 2, 3, 2], [0, 1, 0, ..., 1, 0, 1], ..., [3, 2, 3, ..., 2, 3, 2], [0, 1, 0, ..., 1, 0, 1], [3, 2, 3, ..., 2, 3, 2]], dtype=uint8) raw.raw_colors_visible.shape = (3280, 4948) raw.raw_colors_visible = array([[0, 1, 0, ..., 1, 0, 1], [3, 2, 3, ..., 2, 3, 2], [0, 1, 0, ..., 1, 0, 1], ..., [3, 2, 3, ..., 2, 3, 2], [0, 1, 0, ..., 1, 0, 1], [3, 2, 3, ..., 2, 3, 2]], dtype=uint8) raw.raw_image_visible.shape = (3280, 4948) raw.raw_image_visible.dtype = dtype('uint16') raw.raw_image_visible = array([[ 864, 930, 854, ..., 647, 640, 658], [1062, 308, 977, ..., 219, 746, 219], [ 802, 919, 867, ..., 710, 625, 685], ..., [ 815, 322, 833, ..., 146, 511, 192], [ 769, 726, 751, ..., 513, 466, 468], [ 771, 321, 813, ..., 149, 525, 189]], dtype=uint16) raw.raw_image.min() = 0 raw.raw_image.max() = 4650 raw.raw_image.shape = (3280, 4992) raw.raw_image = array([[ 864, 930, 854, ..., 0, 0, 0], [1062, 308, 977, ..., 0, 0, 0], [ 802, 919, 867, ..., 0, 0, 0], ..., [ 815, 322, 833, ..., 0, 0, 0], [ 769, 726, 751, ..., 0, 0, 0], [ 771, 321, 813, ..., 0, 0, 0]], dtype=uint16) raw.raw_pattern = array([[0, 1], [3, 2]], dtype=uint8) raw.raw_type = <RawType.Flat: 0> raw.rgb_xyz_matrix = array([[ 0.8198, -0.2239, -0.0724], [-0.4871, 1.2389, 0.2798], [-0.1043, 0.205 , 0.7181], [ 0. , 0. , 0. ]], dtype=float32) raw.sizes = ImageSizes(raw_height=3280, raw_width=4992, height=3280, width=4948, top_margin=0, left_margin=0, iheight=3280, iwidth=4948, pixel_aspect=1.0, flip=0) δy = 0 δx = 0 ``` Why different `raw_width` and `width`? `raw_image` seems to have `0` in this case at the right side while it does not in `raw_image_visible`. Could verify color correctness in the RAW array. As in internship offer display image without interpolation would ease doing so it seems, in addition to be interesting. ```py help(rawpy.RawPy.postprocess) help(rawpy.Params) help(rawpy.DemosaicAlgorithm) ``` seem to show that we have to do it on our own. How to compute PRNU on these arrays? Only considering a given color channel seems to make sense. Pay attention that green might have a different shape. More precisely it does not seem to have the grid shape as the 2 other channels. ```py (4992 - 4948) / 2 ``` ``` 22.0 ``` ```py import math math.gcd(3280, 4992) math.gcd(3280, 4948) ``` ``` 16 4 ```
Benjamin_Loison commented 2024-04-15 15:04:49 +02:00
Author
Owner
Copy Link

Raw script:

import rawpy
import numpy as np
from tqdm import tqdm
import matplotlib.pyplot as plt

filename = 'r01b6aed6t.NEF'

colors = {
    'R': (255, 0, 0),
    'G': (0, 255, 0),
    'B': (0, 0, 255),
}

with rawpy.imread(filename) as raw:
    colorDesc = raw.color_desc.decode('ascii')
    colorsIndexes = [colors[color] for color in colorDesc]
    
    raw_colors_visible = raw.raw_colors_visible.copy()
    raw_image_visible = raw.raw_image_visible.copy()
    
    shape = raw_image_visible.shape
    # Consider the maximum value per channel does not help, as they all reach the same maximum value, at least on `r01b6aed6t.NEF`.
    rawImageVisibleMax = raw_image_visible.max()
    image = np.empty((shape[0], shape[1], 3), dtype=np.uint8)
    for y in tqdm(range(shape[0])):
        for x in range(shape[1]):
            value = np.array(colorsIndexes[raw_colors_visible[y][x]]) * raw_image_visible[y][x] / rawImageVisibleMax
            image[y][x] = value

assert colorDesc == 'RGBG'
assert np.array_equal(raw.raw_pattern, np.array([[3, 2], [0, 1]], dtype = np.uint8))

colorImages = {
    'R': image[1::2, ::2],
    'B': image[::2, 1::2],
}

colorImages['G'] = np.array([line[(0 if lineIndex % 2 == 0 else 1)::2] for lineIndex, line in enumerate(image)])

plt.imsave(f'{filename}.png', image)
for color, colorImage in colorImages.items():
    plt.imsave(f'{filename}_{color}.png', colorImage)
Raw script: ```py import rawpy import numpy as np from tqdm import tqdm import matplotlib.pyplot as plt filename = 'r01b6aed6t.NEF' colors = { 'R': (255, 0, 0), 'G': (0, 255, 0), 'B': (0, 0, 255), } with rawpy.imread(filename) as raw: colorDesc = raw.color_desc.decode('ascii') colorsIndexes = [colors[color] for color in colorDesc] raw_colors_visible = raw.raw_colors_visible.copy() raw_image_visible = raw.raw_image_visible.copy() shape = raw_image_visible.shape # Consider the maximum value per channel does not help, as they all reach the same maximum value, at least on `r01b6aed6t.NEF`. rawImageVisibleMax = raw_image_visible.max() image = np.empty((shape[0], shape[1], 3), dtype=np.uint8) for y in tqdm(range(shape[0])): for x in range(shape[1]): value = np.array(colorsIndexes[raw_colors_visible[y][x]]) * raw_image_visible[y][x] / rawImageVisibleMax image[y][x] = value assert colorDesc == 'RGBG' assert np.array_equal(raw.raw_pattern, np.array([[3, 2], [0, 1]], dtype = np.uint8)) colorImages = { 'R': image[1::2, ::2], 'B': image[::2, 1::2], } colorImages['G'] = np.array([line[(0 if lineIndex % 2 == 0 else 1)::2] for lineIndex, line in enumerate(image)]) plt.imsave(f'{filename}.png', image) for color, colorImage in colorImages.items(): plt.imsave(f'{filename}_{color}.png', colorImage) ```
Benjamin_Loison commented 2024-04-15 15:30:51 +02:00
Author
Owner
Copy Link

np.unique(raw_image_visible) shows that values 1, 2 etc are also taken, there is no gap.

`np.unique(raw_image_visible)` shows that values `1`, `2` etc are also taken, there is no gap.
Benjamin_Loison commented 2024-04-15 15:44:05 +02:00
Author
Owner
Copy Link

Taken with Nikon D90:

r01b6aed6t

r01b6aed6t.NEF

The bottom left white part is interesting to see the colored tiles in the three color channels.

At the top image border and mainly at left there are 2 red colored dots.

Red colored artificial flowers on the plates are also interesting to see almost only red tiles.

Increasing brightness and contrast also show the tiles everywhere.

Except if the Bayer filter is particularly involved in the PRNU, the color channel should not matter a lot, see #50. Related to #44.

Taken with `Nikon D90`: ![r01b6aed6t](/attachments/373e7790-0c1c-4c1c-8c5d-80cd651bb6e4) ![r01b6aed6t.NEF](/attachments/f3c0576f-587a-48d6-a8ea-b72aa9283ddb) The bottom left white part is interesting to see the colored tiles in the three color channels. At the top image border and mainly at left there are 2 red colored dots. Red colored artificial flowers on the plates are also interesting to see almost only red tiles. Increasing brightness and contrast also show the tiles everywhere. Except if the Bayer filter is particularly involved in the PRNU, the color channel should not matter a lot, see #50. Related to #44.
r01b6aed6t.png
23 MiB
r01b6aed6t.NEF.png
10 MiB
Benjamin_Loison commented 2024-04-15 20:04:58 +02:00
Author
Owner
Copy Link

r01b6aed6t.NEF_R

r01b6aed6t.NEF_G

r01b6aed6t.NEF_B

![r01b6aed6t.NEF_R](/attachments/bb9f9619-de9f-472d-a93b-8d00fb02dae8) ![r01b6aed6t.NEF_G](/attachments/23223db6-b50f-49d4-a8e3-2b4093e4b5b0) ![r01b6aed6t.NEF_B](/attachments/b12256e5-080b-4e79-bbea-7931266e26d4)
r01b6aed6t.NEF_B.png
2.0 MiB
r01b6aed6t.NEF_R.png
2.0 MiB
r01b6aed6t.NEF_G.png
4.3 MiB
Benjamin_Loison commented 2024-04-15 20:34:52 +02:00
Author
Owner
Copy Link

Let us give a try with red as above red image looks easier to understand than the blue one.

Considering a subset of the green image seems appropriate as well assuming the independence of sensors which is maybe not exactly the case, especially for green ones.

Let us give a try with red as above [red image](https://gitea.lemnoslife.com/Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones/attachments/bb9f9619-de9f-472d-a93b-8d00fb02dae8) looks easier to understand than [the blue one](https://gitea.lemnoslife.com/Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones/attachments/b12256e5-080b-4e79-bbea-7931266e26d4). Considering a subset of the green image seems appropriate as well assuming the independence of sensors which is maybe not exactly the case, especially for green ones.
Benjamin_Loison commented 2024-04-15 21:38:21 +02:00
Author
Owner
Copy Link

Can verify the consistency of the subset of pixels by verifying areas using as most as possible a single color channel.

search_green.py:

#!/usr/bin/python3

from PIL import Image
import numpy as np
from tqdm import tqdm
import os

GREEN = np.array([0, 255, 0])

minimalDistance = None
minimalDistanceLocation = None

folder = 'raise_nef_png'

for file in tqdm(os.listdir(folder), 'Files'):
    with Image.open(f'{folder}/{file}') as image:
        imageNpArray = np.array(image)
        imageNpArrayShape = imageNpArray.shape
        for y in tqdm(range(imageNpArrayShape[0]), file):
            for x in range(imageNpArrayShape[1]):
                value = imageNpArray[y][x]
                distance = np.linalg.norm(value - GREEN)
                if minimalDistance is None or distance < minimalDistance:
                    print(file, minimalDistance, minimalDistanceLocation)
                    minimalDistance = distance
                    minimalDistanceLocation = (y, x)

ccd17f6bt.png 57.14017850864661 (1243, 3526)
...
r1a9acd94t.png: 100%|███████████████████████████████████████████████████████████| 4310/4310 [00:42<00:00, 100.42it/s]
Files:   2%|█▍                                                              | 175/8076 [2:35:00<113:59:13, 51.94s/it
...

Could iterate on images to find the most clear one.

Pay attention to possible visible offset.

Can verify the consistency of the subset of pixels by verifying areas using as most as possible a single color channel. `search_green.py`: ```py #!/usr/bin/python3 from PIL import Image import numpy as np from tqdm import tqdm import os GREEN = np.array([0, 255, 0]) minimalDistance = None minimalDistanceLocation = None folder = 'raise_nef_png' for file in tqdm(os.listdir(folder), 'Files'): with Image.open(f'{folder}/{file}') as image: imageNpArray = np.array(image) imageNpArrayShape = imageNpArray.shape for y in tqdm(range(imageNpArrayShape[0]), file): for x in range(imageNpArrayShape[1]): value = imageNpArray[y][x] distance = np.linalg.norm(value - GREEN) if minimalDistance is None or distance < minimalDistance: print(file, minimalDistance, minimalDistanceLocation) minimalDistance = distance minimalDistanceLocation = (y, x) ``` ``` ccd17f6bt.png 57.14017850864661 (1243, 3526) ... r1a9acd94t.png: 100%|███████████████████████████████████████████████████████████| 4310/4310 [00:42<00:00, 100.42it/s] Files: 2%|█▍ | 175/8076 [2:35:00<113:59:13, 51.94s/it ... ``` Could iterate on images to find the most clear one. Pay attention to possible `visible` offset.
Benjamin_Loison commented 2024-04-15 23:40:50 +02:00
Author
Owner
Copy Link

Related to Benjamin_Loison/rawpy/issues/1.

Related to [Benjamin_Loison/rawpy/issues/1](https://codeberg.org/Benjamin_Loison/rawpy/issues/1).
Benjamin_Loison commented 2024-04-16 00:42:10 +02:00
Author
Owner
Copy Link
help(rawpy.RawPy.raw_pattern)

Help on getset descriptor rawpy._rawpy.RawPy.raw_pattern:

raw_pattern
    The smallest possible Bayer pattern of this image.
    
    :rtype: ndarray, or None if not a flat RAW image

help(rawpy.RawPy.raw_colors_visible)

Help on getset descriptor rawpy._rawpy.RawPy.raw_colors_visible:

raw_colors_visible
    Like raw_colors but without margin.
    
    :rtype: ndarray of shape (hv,wv)

help(rawpy.RawPy.raw_colors)

Help on getset descriptor rawpy._rawpy.RawPy.raw_colors:

raw_colors
    An array of color indices for each pixel in the RAW image.
    Equivalent to calling raw_color(y,x) for each pixel.
    Only usable for flat RAW images (see raw_type property).
    
    :rtype: ndarray of shape (h,w)

help(rawpy.RawPy.raw_color)

Help on cython_function_or_method in module rawpy._rawpy:

raw_color(self, row, column)
    RawPy.raw_color(self, int row, int column) -> int
    
    Return color index for the given coordinates relative to the full RAW size.
    Only usable for flat RAW images (see raw_type property).

help(rawpy.RawPy.color_desc)

Help on getset descriptor rawpy._rawpy.RawPy.color_desc:

color_desc
    String description of colors numbered from 0 to 3 (RGBG,RGBE,GMCY, or GBTG).
    Note that same letters may not refer strictly to the same color.
    There are cameras with two different greens for example.

help(rawpy.RawPy.color_matrix)

Help on getset descriptor rawpy._rawpy.RawPy.color_matrix:

color_matrix
    Color matrix, read from file for some cameras, calculated for others. 
    
    :rtype: ndarray of shape (3,4)
```py help(rawpy.RawPy.raw_pattern) ``` ``` Help on getset descriptor rawpy._rawpy.RawPy.raw_pattern: raw_pattern The smallest possible Bayer pattern of this image. :rtype: ndarray, or None if not a flat RAW image ``` ```py help(rawpy.RawPy.raw_colors_visible) ``` ``` Help on getset descriptor rawpy._rawpy.RawPy.raw_colors_visible: raw_colors_visible Like raw_colors but without margin. :rtype: ndarray of shape (hv,wv) ``` ```py help(rawpy.RawPy.raw_colors) ``` ``` Help on getset descriptor rawpy._rawpy.RawPy.raw_colors: raw_colors An array of color indices for each pixel in the RAW image. Equivalent to calling raw_color(y,x) for each pixel. Only usable for flat RAW images (see raw_type property). :rtype: ndarray of shape (h,w) ``` ```py help(rawpy.RawPy.raw_color) ``` ``` Help on cython_function_or_method in module rawpy._rawpy: raw_color(self, row, column) RawPy.raw_color(self, int row, int column) -> int Return color index for the given coordinates relative to the full RAW size. Only usable for flat RAW images (see raw_type property). ``` ```py help(rawpy.RawPy.color_desc) ``` ``` Help on getset descriptor rawpy._rawpy.RawPy.color_desc: color_desc String description of colors numbered from 0 to 3 (RGBG,RGBE,GMCY, or GBTG). Note that same letters may not refer strictly to the same color. There are cameras with two different greens for example. ``` ```py help(rawpy.RawPy.color_matrix) ``` ``` Help on getset descriptor rawpy._rawpy.RawPy.color_matrix: color_matrix Color matrix, read from file for some cameras, calculated for others. :rtype: ndarray of shape (3,4) ```
Benjamin_Loison commented 2024-04-16 00:48:50 +02:00
Author
Owner
Copy Link

Related to Benjamin_Loison/vpv/issues/3 and Benjamin_Loison/darktable/issues/3.

Related to [Benjamin_Loison/vpv/issues/3](https://gitea.lemnoslife.com/Benjamin_Loison/vpv/issues/3) and [Benjamin_Loison/darktable/issues/3](https://codeberg.org/Benjamin_Loison/darktable/issues/3).
Benjamin_Loison commented 2024-04-16 01:11:51 +02:00
Author
Owner
Copy Link

Would be nice to not have to assert raw.raw_pattern.

Would be nice to not have to `assert` `raw.raw_pattern`.
Benjamin_Loison commented 2024-04-16 01:14:29 +02:00
Author
Owner
Copy Link

ra2c888f8t was done with Nikon D7000.

`ra2c888f8t` was done with Nikon D7000.
Benjamin_Loison commented 2024-04-16 01:16:20 +02:00
Author
Owner
Copy Link
import rawpy
import numpy as np
from tqdm import tqdm
import matplotlib.pyplot as plt

filename = 'ra2c888f8t.NEF'

colors = {
    'R': (255, 0, 0),
    'G': (0, 255, 0),
    'B': (0, 0, 255),
}

with rawpy.imread(filename) as raw:
    colorDesc = raw.color_desc.decode('ascii')
    assert colorDesc == 'RGBG'
    assert np.array_equal(raw.raw_pattern, np.array([[0, 1], [3, 2]], dtype = np.uint8))
    colorsIndexes = [colors[color] for color in colorDesc]
    
    raw_colors_visible = raw.raw_colors_visible.copy()
    raw_image_visible = raw.raw_image_visible.copy()
    
    shape = raw_image_visible.shape
    # Consider the maximum value per channel does not help, as they all reach the same maximum value, at least on `r01b6aed6t.NEF`.
    rawImageVisibleMax = raw_image_visible.max()
    image = np.empty((shape[0], shape[1], 3), dtype=np.uint8)
    for y in tqdm(range(shape[0])):
        for x in range(shape[1]):
            value = np.array(colorsIndexes[raw_colors_visible[y][x]]) * raw_image_visible[y][x] / rawImageVisibleMax
            image[y][x] = value

colorImages = {
    'R': image[::2, ::2],
    'G_S': image[1::2, ::2],
    'B': image[1::2, 1::2],
}

colorImages['G'] = np.array([line[(0 if lineIndex % 2 == 1 else 1)::2] for lineIndex, line in enumerate(image)])

plt.imsave(f'{filename}.png', image)
for color, colorImage in colorImages.items():
    plt.imsave(f'{filename}_{color}.png', colorImage)
```py import rawpy import numpy as np from tqdm import tqdm import matplotlib.pyplot as plt filename = 'ra2c888f8t.NEF' colors = { 'R': (255, 0, 0), 'G': (0, 255, 0), 'B': (0, 0, 255), } with rawpy.imread(filename) as raw: colorDesc = raw.color_desc.decode('ascii') assert colorDesc == 'RGBG' assert np.array_equal(raw.raw_pattern, np.array([[0, 1], [3, 2]], dtype = np.uint8)) colorsIndexes = [colors[color] for color in colorDesc] raw_colors_visible = raw.raw_colors_visible.copy() raw_image_visible = raw.raw_image_visible.copy() shape = raw_image_visible.shape # Consider the maximum value per channel does not help, as they all reach the same maximum value, at least on `r01b6aed6t.NEF`. rawImageVisibleMax = raw_image_visible.max() image = np.empty((shape[0], shape[1], 3), dtype=np.uint8) for y in tqdm(range(shape[0])): for x in range(shape[1]): value = np.array(colorsIndexes[raw_colors_visible[y][x]]) * raw_image_visible[y][x] / rawImageVisibleMax image[y][x] = value colorImages = { 'R': image[::2, ::2], 'G_S': image[1::2, ::2], 'B': image[1::2, 1::2], } colorImages['G'] = np.array([line[(0 if lineIndex % 2 == 1 else 1)::2] for lineIndex, line in enumerate(image)]) plt.imsave(f'{filename}.png', image) for color, colorImage in colorImages.items(): plt.imsave(f'{filename}_{color}.png', colorImage) ```
Benjamin_Loison pinned this 2024-04-16 01:39:56 +02:00
Benjamin_Loison commented 2024-04-16 01:40:15 +02:00
Author
Owner
Copy Link

Could consider the channel having the most variance on the considered set of images.

Could consider the channel having the most variance on the considered set of images.
Benjamin_Loison commented 2024-04-16 02:52:55 +02:00
Author
Owner
Copy Link

Are raw values bounded? Should check other rawpy.RawPy elements and compute statistics on a given image.

Are raw values bounded? Should check other `rawpy.RawPy` elements and compute statistics on a given image.
Benjamin_Loison commented 2024-04-16 03:02:13 +02:00
Author
Owner
Copy Link

With wavelet denoiser:

np_array

image

With bilateral denoiser:

Computing extremes of images: 100%|████████████████████████████████████████████████| 100/100 [00:32<00:00,  3.05it/s]
Denoising images:   8%|████▋                                                      | 8/100 [36:40<7:01:44, 275.05s/it]Denoising images: 100%|█████████████████████████████████████████████████████████| 100/100 [7:38:12<00:00, 274.93s/it]

np_array

np_array_1

With wavelet denoiser: ![np_array](/attachments/caa24132-03ce-4e6b-82a3-818d87b1694f) ![image](/attachments/8ad321f0-6608-40c1-9000-7090dd8bd7a0) With bilateral denoiser: ``` Computing extremes of images: 100%|████████████████████████████████████████████████| 100/100 [00:32<00:00, 3.05it/s] Denoising images: 8%|████▋ | 8/100 [36:40<7:01:44, 275.05s/it]Denoising images: 100%|█████████████████████████████████████████████████████████| 100/100 [7:38:12<00:00, 274.93s/it] ``` ![np_array](/attachments/4e5df657-c198-48fb-a60d-7a696d96a7d2) ![np_array_1](/attachments/5d2afc52-b018-423a-96d2-0a96b2678129)
image.png
1.4 MiB
np_array.png
7.9 MiB
np_array.png
5.9 MiB
np_array_1.png
1.0 MiB
Benjamin_Loison commented 2024-04-16 03:10:52 +02:00
Author
Owner
Copy Link

Contrairement aux canaux rouge et bleu, l’utilisation du canal vert pour l’estimation du PRNU semble incertaine à cause de la répartition du vert. Est-ce qu’un décalage dans une des deux directions a vraiment une influence significative ?

> Contrairement aux canaux rouge et bleu, l’utilisation du canal vert pour l’estimation du PRNU semble incertaine à cause de la répartition du vert. Est-ce qu’un décalage dans une des deux directions a vraiment une influence significative ?
Benjamin_Loison commented 2024-04-16 03:38:11 +02:00
Author
Owner
Copy Link

Rafael ARW sky wavelet:

np_array

Rafael ARW wall wavelet:

image

image

I do not seem to find any visual feature in both. Except maybe the latter with an horizontal line at the bottom and possibly a lens circle at the top and bottom right.

Rafael ARW sky bilateral:

mean_rafael_arw_bilateral_green_right

image

Computing extremes of images: 100%|████████████████████████████████████████████████| 108/108 [00:17<00:00,  6.12it/s]
minColor=1080
maxColor=6556
Denoising images for color green_right: 100%|██████████████████████████████████| 108/108 [12:20:38<00:00, 411.47s/it]

Rafael ARW wall bilateral:

image

image

Rafael ARW sky wavelet: ![np_array](/attachments/6a087278-77ac-4a0b-b471-882054563238) Rafael ARW wall wavelet: ![image](/attachments/f7299848-2935-44f5-8c70-10011f713dac) ![image](/attachments/27ae7502-b037-4056-b527-53b582004c62) I do not seem to find any visual feature in both. Except maybe the latter with an horizontal line at the bottom and possibly a lens circle at the top and bottom right. Rafael ARW sky bilateral: ![mean_rafael_arw_bilateral_green_right](/attachments/420743a1-0623-4c7e-8062-248e1d111b59) ![image](/attachments/351c5137-e8d3-4b75-a684-ae52f5fda4d0) ``` Computing extremes of images: 100%|████████████████████████████████████████████████| 108/108 [00:17<00:00, 6.12it/s] minColor=1080 maxColor=6556 Denoising images for color green_right: 100%|██████████████████████████████████| 108/108 [12:20:38<00:00, 411.47s/it] ``` Rafael ARW wall bilateral: ![image](/attachments/fcf19526-f425-4c16-b497-8ed26acfa794) ![image](/attachments/52245566-cb6b-41ff-9494-eb7f740abe82)
np_array.png
13 MiB
image.png
11 MiB
image.png
800 KiB
image.png
10 MiB
image.png
2.1 MiB
mean_rafael_arw_bilateral_green_right.png
11 MiB
image.png
1.8 MiB
Benjamin_Loison commented 2024-04-16 04:25:44 +02:00
Author
Owner
Copy Link

Should implement estimating the PRNU for the 3 color channels. Considering a raw image as if not interpolating does not seem correct despite each color channel image having the same size as others, because the resulting pixel does not have the same light trajectory.

Just for visualizing can do so but otherwise should compute the distance as the sum for each color channel or regular euclidian for instance?

Should implement estimating the PRNU for the 3 color channels. Considering a raw image as if not interpolating does not seem correct despite each color channel image having the same size as others, because the resulting pixel does not have the same light trajectory. Just for visualizing can do so but otherwise should compute the distance as the sum for each color channel or regular euclidian for instance?
Benjamin_Loison commented 2024-04-16 19:30:04 +02:00
Author
Owner
Copy Link

As requested by Marina:

flat_001

flat_001.NEF

flat_001.NEF_B

flat_001.NEF_G

flat_001.NEF_G_S

flat_001.NEF_R

As requested by Marina: ![flat_001](/attachments/3bf61c40-c4a5-402e-b5c2-022c0b1043d8) ![flat_001.NEF](/attachments/02c39842-4eb1-4612-938a-1bd728ccbecd) ![flat_001.NEF_B](/attachments/c7e080e0-8b68-43a8-a638-629db4137cb9) ![flat_001.NEF_G](/attachments/c23c2af9-bc55-46bb-8a95-bbcb7b870bd9) ![flat_001.NEF_G_S](/attachments/cd8a69fd-fb4f-4946-a01f-ebec0b24f3a0) ![flat_001.NEF_R](/attachments/38d3b4c6-dea6-4bea-b058-626085813fef)
flat_001.NEF.png
15 MiB
flat_001.NEF_B.png
3.0 MiB
flat_001.NEF_G.png
6.7 MiB
flat_001.NEF_G_S.png
3.3 MiB
flat_001.NEF_R.png
3.4 MiB
flat_001.NEF
19 MiB
Benjamin_Loison commented 2024-04-18 00:14:53 +02:00
Author
Owner
Copy Link

Concerning merging multiple color channels PRNU estimations, I choose to input multiple images instead of multiple Numpy arrays, as I may adjust with GIMP to have brightness and contrast settings easing the visualization.

Il y aura probablement un problème d’échelles entre les canaux puisque deux pixels adjacents de couleurs différentes vont enregistrer des intensités potentiellement très différentes.

Flat-field wavelet:

multipleColors

Improved visualization thanks to brightness and contrast change, only contrast if I remember correctly:

image

Flat-field bilateral:

multipleColors

multipleColors_brightness_contrast_modified

For unclear reasons the colors of green right and bottom differ quite significantly, displaying the actual distribution would help understand.

Concerning merging multiple color channels PRNU estimations, I choose to input multiple images instead of multiple Numpy arrays, as I may adjust with GIMP to have brightness and contrast settings easing the visualization. Il y aura probablement un problème d’échelles entre les canaux puisque deux pixels adjacents de couleurs différentes vont enregistrer des intensités potentiellement très différentes. Flat-field wavelet: ![multipleColors](/attachments/b578c77b-ee85-4712-8b9b-e925b27062e6) Improved visualization thanks to brightness and contrast change, only contrast if I remember correctly: ![image](/attachments/96f5be42-61f7-41f2-b1f4-b3e73b65d000) Flat-field bilateral: ![multipleColors](/attachments/e9809fcc-1cb2-43a6-bff5-80972959cd42) ![multipleColors_brightness_contrast_modified](/attachments/c21a6483-eab8-4a7d-8c7c-20159dcc93cc) For unclear reasons the colors of green right and bottom differ quite significantly, displaying the actual distribution would help understand.
image.png
3.8 MiB
multipleColors.png
33 MiB
multipleColors.png
23 MiB
multipleColors_brightness_contrast_modified.png
1.8 MiB
Benjamin_Loison commented 2024-04-19 16:53:44 +02:00
Author
Owner
Copy Link

Correctly assembled 4 PRNUs:

final

Do not have lines anymore, as do not - npArrayMin, as I thought it was to make matplotlib happy but it does not seem to care and it only introduce bias by introducing a different offset per color channel.

Also note that we do not notice the Bayer filter pattern, i.e. red being particularly present as the wall look orange for instance, as we are considering noise for each color channel, so the resulting image is just about difference in each color channel which is more similar than absolute values of each color channel.

Improved brightness and contrast:

image

Correctly assembled 4 PRNUs: ![final](/attachments/61060c5a-b06d-4953-8340-4915af281356) Do not have lines anymore, as do not `- npArrayMin`, as I thought it was to make matplotlib happy but it does not seem to care and it only introduce bias by introducing a different offset per color channel. Also note that we do not notice the Bayer filter pattern, i.e. red being particularly present as the wall look orange for instance, as we are considering noise for each color channel, so the resulting image is just about difference in each color channel which is more similar than absolute values of each color channel. Improved brightness and contrast: ![image](/attachments/b5fc741e-d52f-4afb-86fb-71905973b60d)
final.png
18 MiB
image.png
4.8 MiB
Benjamin_Loison commented 2024-04-21 19:47:21 +02:00
Author
Owner
Copy Link

4 PRNUs bilateral denoiser assembled for Rafael wall:

multiple_colors

Modified brightness and contrast:

image

Do we also notice lines with wavelet denoiser and on sky too?

Seems to have similar behavior with Rafael sky with mean denoiser (related to #57):

mean_rafael_arw_sky_mean_multiple_colors

Modified brightness and contrast:

image

Similar result with the wall:

mean_rafael_arw_wall_mean_multiple_colors

Could upload 4 PRNUs bilateral denoiser assembled for Rafael sky.

Why is the Bayer filter that visible?

First image histogram:

first_image

Mean of images histogram:

mean

First Bayer filter occurrence histogram:

first_bayer_filter_occurrence

First Bayer filter occurrence minus mean of images histogram:

first_bayer_filter_occurrence_minus_mean

RAISE flat-field incorrect PRNU estimation with mean denoiser (the PRNU estimation is just the mean of images) with 4 color channels assembled:

mean_flat-field_NEF_mean_multiple_colors

With modified brightness and contrast:

mean_flat-field_NEF_mean_multiple_colors_brightness_contrast_changed

The respective corrected images:

mean_flat-field_NEF_mean_multiple_colors

image

Annotated flat_099.tif showing some circles on the camera.

flat_099_annotated

center_wall_marker_locations

distance_between_a_wall_marker_location_and_the_next_one

RAISE flat-field timeline:

raise_flat-field_timeline

Rafael 24/04/24:

rafael_240424

3064 to 3078:

rafael_240424_3064_to_3078

22/04/24:

rafael_240424_220424

23/04/24:

rafael_240424_230424

4 PRNUs bilateral denoiser assembled for Rafael wall: ![multiple_colors](/attachments/b07c6d3a-97a6-474f-988b-b5609b06d15c) Modified brightness and contrast: ![image](/attachments/b5641a49-8621-454a-a44d-bddd65b28a89) Do we also notice lines with wavelet denoiser and on sky too? Seems to have similar behavior with Rafael sky with mean denoiser (related to #57): ![mean_rafael_arw_sky_mean_multiple_colors](/attachments/84be6353-da0e-443f-83b8-61076b5ac44b) Modified brightness and contrast: ![image](/attachments/99d8c95a-dc48-4c3b-9539-2b344a9c7da0) Similar result with the wall: ![mean_rafael_arw_wall_mean_multiple_colors](/attachments/88d928fd-3093-4081-b809-6440e702402a) Could upload 4 PRNUs bilateral denoiser assembled for Rafael sky. Why is the Bayer filter that visible? First image histogram: ![first_image](/attachments/5e84dfa5-de1d-49d3-9d0f-93a55000817d) Mean of images histogram: ![mean](/attachments/1e6800a0-32bf-442d-800a-91569b7438d1) First Bayer filter occurrence histogram: ![first_bayer_filter_occurrence](/attachments/c781c784-d6d8-42b1-aeab-b13a9e39904d) First Bayer filter occurrence minus mean of images histogram: ![first_bayer_filter_occurrence_minus_mean](/attachments/13dae3c7-d4ba-490e-9084-36a7ca03766d) RAISE flat-field incorrect PRNU estimation with mean denoiser (the PRNU estimation is just the mean of images) with 4 color channels assembled: ![mean_flat-field_NEF_mean_multiple_colors](/attachments/934e0bcd-e4bf-43a1-b2b5-5e0e4c96bb02) With modified brightness and contrast: ![mean_flat-field_NEF_mean_multiple_colors_brightness_contrast_changed](/attachments/6891c748-58fe-4264-95ec-2a639319f3d3) The respective corrected images: ![mean_flat-field_NEF_mean_multiple_colors](/attachments/cec62f9b-37a3-45cf-be07-c5b5b943adf8) ![image](/attachments/8c976d3f-f0a7-4e28-b285-1e0c2fda7802) Annotated `flat_099.tif` showing some circles on the camera. ![flat_099_annotated](/attachments/47711922-f231-4707-a6a7-b7497fa4b499) ![center_wall_marker_locations](/attachments/c1e7e78b-6eb8-4d35-b3fc-26997a72382f) ![distance_between_a_wall_marker_location_and_the_next_one](/attachments/ead76174-b7bb-47c5-9676-c2af5a8992e9) RAISE flat-field timeline: ![raise_flat-field_timeline](/attachments/19cdbc12-d1c5-4914-b16a-ef610a59bcdd) Rafael 24/04/24: ![rafael_240424](/attachments/505577f2-29b2-4703-a870-ccb6784e0849) 3064 to 3078: ![rafael_240424_3064_to_3078](/attachments/eed6aa84-95e4-4e53-941d-b7a5376d7100) 22/04/24: ![rafael_240424_220424](/attachments/9b890f5c-3ced-4206-9075-eb5c8d3f2e9b) 23/04/24: ![rafael_240424_230424](/attachments/5fca61e7-0781-487a-aeba-f113475d1bb0)
multiple_colors.png
30 MiB
image.png
3.3 MiB
image.png
980 KiB
mean_rafael_arw_sky_mean_multiple_colors.png
25 MiB
mean_rafael_arw_wall_mean_multiple_colors.png
37 MiB
first_image.svg
170 KiB
mean.svg
167 KiB
first_bayer_filter_occurrence.svg
64 KiB
first_bayer_filter_occurrence_minus_mean.svg
63 KiB
mean_flat-field_NEF_mean_multiple_colors.png
30 MiB
mean_flat-field_NEF_mean_multiple_colors_brightness_contrast_changed.png
2.8 MiB
flat_099_annotated.tif
46 MiB
center_wall_marker_locations.svg
41 KiB
distance_between_a_wall_marker_location_and_the_next_one.svg
24 KiB
raise_flat-field_timeline.svg
268 KiB
rafael_240424.svg
395 KiB
rafael_240424_3064_to_3078.svg
129 KiB
rafael_240424_220424.svg
204 KiB
rafael_240424_230424.png
277 KiB
mean_flat-field_NEF_mean_multiple_colors.png
6.0 MiB
image.png
6.1 MiB
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
Context-Adaptive Interpolator

   
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
epic

A task that is going to take more than a day to complete.

  
help wanted

Need some help

  
high priority

   
invalid

Something is wrong

  
left for future work

   
low priority

   
medium

A task that should take less than a day to complete.

  
medium priority

   
meta

   
question

More information is needed

  
quick

A task that should take less than two hours to complete.

  
wontfix

This won't be fixed

No Label
bug
Context-Adaptive Interpolator
duplicate
enhancement
epic
help wanted
high priority
invalid
left for future work
low priority
medium
medium priority
meta
question
quick
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones#49
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?