RAW image processing #49

New Issue

Open

opened 2024-04-15 11:34:15 +02:00 by Benjamin_Loison · 23 comments

Benjamin_Loison commented 2024-04-15 11:34:15 +02:00

Owner

Copy Link

Copy Source

import rawpy

help(rawpy)

help(rawpy)
Help on package rawpy:

NAME
    rawpy

PACKAGE CONTENTS
    _rawpy
    _version
    enhance

FUNCTIONS
    imread(pathOrFile)
        Convenience function that creates a :class:`rawpy.RawPy` instance, opens the given file,
        and returns the :class:`rawpy.RawPy` instance for further processing.
        
        :param str|file pathOrFile: path or file object of RAW image that will be read
        :rtype: :class:`rawpy.RawPy`

DATA
    absolute_import = _Feature((2, 5, 0, 'alpha', 1), (3, 0, 0, 'alpha', 0...
    flags = {'6BY9RPI': True, 'DEMOSAIC_PACK_GPL2': False, 'DEMOSAIC_PACK_...
    libraw_version = (0, 21, 1)

VERSION
    0.19.1

FILE
    /home/benjamin/.local/lib/python3.10/site-packages/rawpy/__init__.py

filename = 'flat_001.NEF'

raw = rawpy.imread(filename)
help(raw)

...
class RawPy(builtins.object)
 |  Load RAW images, work on their data, and create a postprocessed (demosaiced) image.
...
 |  close(self)
...
 |  postprocess(self, params=None, **kw)
...
 |  raw_color(self, row, column)
...
 |  raw_value(self, row, column)
...
 |  raw_value_visible(self, row, column)
...
 |  color_desc
...
 |  color_matrix
...
 |  num_colors
...
 |  raw_colors
...
 |  raw_image
...
 |  raw_pattern
...
 |  raw_type
...
 |  rgb_xyz_matrix
...
 |  sizes

skipping low-level methods proposing an alternative.

Do not get _visible meaning.

import rawpy

raw = rawpy.imread(filename)

data = raw.raw_image_visible.astype(np.float64)
δy, δx = np.argwhere(raw.raw_colors == 0)[0]  # offset on the raw
# red channel corresponds to data[δy::2, δx::2]
# blue channel corresponds to data[1 - δy::2, 1 - δx::2]
# first green channel corresponds to data[δy::2, 1 - δx::2]
# second green channel corresponds to data[1 - δy::2, δx::2]

import imageio

with rawpy.imread(filename) as raw:
    rgb = raw.postprocess()

imageio.imsave('default.tiff', rgb)

Note that according to the Stack Overflow answer 51304155 _visible array versions are not contiguous, hence imply very significant performance decrease. It seems that OWNDATA flag is also involved:

>>> raw.raw_colors_visible.flags
  C_CONTIGUOUS : False
  F_CONTIGUOUS : False
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False


>>> raw.raw_colors_visible.copy().flags
  C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False

```py import rawpy help(rawpy) ``` ``` help(rawpy) Help on package rawpy: NAME rawpy PACKAGE CONTENTS _rawpy _version enhance FUNCTIONS imread(pathOrFile) Convenience function that creates a :class:`rawpy.RawPy` instance, opens the given file, and returns the :class:`rawpy.RawPy` instance for further processing. :param str|file pathOrFile: path or file object of RAW image that will be read :rtype: :class:`rawpy.RawPy` DATA absolute_import = _Feature((2, 5, 0, 'alpha', 1), (3, 0, 0, 'alpha', 0... flags = {'6BY9RPI': True, 'DEMOSAIC_PACK_GPL2': False, 'DEMOSAIC_PACK_... libraw_version = (0, 21, 1) VERSION 0.19.1 FILE /home/benjamin/.local/lib/python3.10/site-packages/rawpy/__init__.py ``` ```py filename = 'flat_001.NEF' raw = rawpy.imread(filename) help(raw) ``` ``` ... class RawPy(builtins.object) | Load RAW images, work on their data, and create a postprocessed (demosaiced) image. ... | close(self) ... | postprocess(self, params=None, **kw) ... | raw_color(self, row, column) ... | raw_value(self, row, column) ... | raw_value_visible(self, row, column) ... | color_desc ... | color_matrix ... | num_colors ... | raw_colors ... | raw_image ... | raw_pattern ... | raw_type ... | rgb_xyz_matrix ... | sizes ``` skipping *low-level methods* proposing an alternative. Do not get `_visible` meaning. ```py import rawpy raw = rawpy.imread(filename) data = raw.raw_image_visible.astype(np.float64) δy, δx = np.argwhere(raw.raw_colors == 0)[0] # offset on the raw # red channel corresponds to data[δy::2, δx::2] # blue channel corresponds to data[1 - δy::2, 1 - δx::2] # first green channel corresponds to data[δy::2, 1 - δx::2] # second green channel corresponds to data[1 - δy::2, δx::2] ``` ```py import imageio with rawpy.imread(filename) as raw: rgb = raw.postprocess() imageio.imsave('default.tiff', rgb) ``` Note that according to [the Stack Overflow answer 51304155](https://stackoverflow.com/a/51304155) `_visible` array versions are not contiguous, hence imply very significant performance decrease. It seems that `OWNDATA` flag is also involved: ```py >>> raw.raw_colors_visible.flags C_CONTIGUOUS : False F_CONTIGUOUS : False OWNDATA : False WRITEABLE : True ALIGNED : True WRITEBACKIFCOPY : False >>> raw.raw_colors_visible.copy().flags C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True WRITEBACKIFCOPY : False ```

Benjamin_Loison added the enhancementmedium prioritymedium labels 2024-04-15 11:34:15 +02:00

Benjamin_Loison commented 2024-04-15 12:08:10 +02:00

Author

Owner

Copy Link

Copy Source

def printWithName(variable):
    print(variable, '=', repr(eval(variable)))

with rawpy.imread(filename) as raw:
    printWithName('raw.color_desc')
    printWithName('raw.color_matrix')
    printWithName('raw.num_colors')
    print()
    
    printWithName('raw.raw_colors.shape')
    printWithName('raw.raw_colors')
    print()
    
    printWithName('raw.raw_colors_visible.shape')
    printWithName('raw.raw_colors_visible')
    print()
    
    printWithName('raw.raw_image_visible.shape')
    printWithName('raw.raw_image_visible.dtype')
    printWithName('raw.raw_image_visible')
    print()
    
    printWithName('raw.raw_image.min()')
    printWithName('raw.raw_image.max()')
    printWithName('raw.raw_image.shape')
    printWithName('raw.raw_image')
    print()
    
    printWithName('raw.raw_pattern')
    printWithName('raw.raw_type')
    printWithName('raw.rgb_xyz_matrix')
    printWithName('raw.sizes')
    print()
    
    δy, δx = np.argwhere(raw.raw_colors == 0)[0]
    printWithName('δy')
    printWithName('δx')

raw.color_desc = b'RGBG'
raw.color_matrix = array([[0., 0., 0., 0.],
       [0., 0., 0., 0.],
       [0., 0., 0., 0.]], dtype=float32)
raw.num_colors = 3

raw.raw_colors.shape = (3280, 4992)
raw.raw_colors = array([[0, 1, 0, ..., 1, 0, 1],
       [3, 2, 3, ..., 2, 3, 2],
       [0, 1, 0, ..., 1, 0, 1],
       ...,
       [3, 2, 3, ..., 2, 3, 2],
       [0, 1, 0, ..., 1, 0, 1],
       [3, 2, 3, ..., 2, 3, 2]], dtype=uint8)

raw.raw_colors_visible.shape = (3280, 4948)
raw.raw_colors_visible = array([[0, 1, 0, ..., 1, 0, 1],
       [3, 2, 3, ..., 2, 3, 2],
       [0, 1, 0, ..., 1, 0, 1],
       ...,
       [3, 2, 3, ..., 2, 3, 2],
       [0, 1, 0, ..., 1, 0, 1],
       [3, 2, 3, ..., 2, 3, 2]], dtype=uint8)

raw.raw_image_visible.shape = (3280, 4948)
raw.raw_image_visible.dtype = dtype('uint16')
raw.raw_image_visible = array([[ 864,  930,  854, ...,  647,  640,  658],
       [1062,  308,  977, ...,  219,  746,  219],
       [ 802,  919,  867, ...,  710,  625,  685],
       ...,
       [ 815,  322,  833, ...,  146,  511,  192],
       [ 769,  726,  751, ...,  513,  466,  468],
       [ 771,  321,  813, ...,  149,  525,  189]], dtype=uint16)

raw.raw_image.min() = 0
raw.raw_image.max() = 4650
raw.raw_image.shape = (3280, 4992)
raw.raw_image = array([[ 864,  930,  854, ...,    0,    0,    0],
       [1062,  308,  977, ...,    0,    0,    0],
       [ 802,  919,  867, ...,    0,    0,    0],
       ...,
       [ 815,  322,  833, ...,    0,    0,    0],
       [ 769,  726,  751, ...,    0,    0,    0],
       [ 771,  321,  813, ...,    0,    0,    0]], dtype=uint16)

raw.raw_pattern = array([[0, 1],
       [3, 2]], dtype=uint8)
raw.raw_type = <RawType.Flat: 0>
raw.rgb_xyz_matrix = array([[ 0.8198, -0.2239, -0.0724],
       [-0.4871,  1.2389,  0.2798],
       [-0.1043,  0.205 ,  0.7181],
       [ 0.    ,  0.    ,  0.    ]], dtype=float32)
raw.sizes = ImageSizes(raw_height=3280, raw_width=4992, height=3280, width=4948, top_margin=0, left_margin=0, iheight=3280, iwidth=4948, pixel_aspect=1.0, flip=0)

δy = 0
δx = 0

Why different raw_width and width?

raw_image seems to have 0 in this case at the right side while it does not in raw_image_visible.

Could verify color correctness in the RAW array. As in internship offer display image without interpolation would ease doing so it seems, in addition to be interesting.

help(rawpy.RawPy.postprocess)
help(rawpy.Params)
help(rawpy.DemosaicAlgorithm)

seem to show that we have to do it on our own.

How to compute PRNU on these arrays? Only considering a given color channel seems to make sense. Pay attention that green might have a different shape. More precisely it does not seem to have the grid shape as the 2 other channels.

(4992 - 4948) / 2

22.0

import math

math.gcd(3280, 4992)
math.gcd(3280, 4948)

16
4

```py def printWithName(variable): print(variable, '=', repr(eval(variable))) with rawpy.imread(filename) as raw: printWithName('raw.color_desc') printWithName('raw.color_matrix') printWithName('raw.num_colors') print() printWithName('raw.raw_colors.shape') printWithName('raw.raw_colors') print() printWithName('raw.raw_colors_visible.shape') printWithName('raw.raw_colors_visible') print() printWithName('raw.raw_image_visible.shape') printWithName('raw.raw_image_visible.dtype') printWithName('raw.raw_image_visible') print() printWithName('raw.raw_image.min()') printWithName('raw.raw_image.max()') printWithName('raw.raw_image.shape') printWithName('raw.raw_image') print() printWithName('raw.raw_pattern') printWithName('raw.raw_type') printWithName('raw.rgb_xyz_matrix') printWithName('raw.sizes') print() δy, δx = np.argwhere(raw.raw_colors == 0)[0] printWithName('δy') printWithName('δx') ``` ``` raw.color_desc = b'RGBG' raw.color_matrix = array([[0., 0., 0., 0.], [0., 0., 0., 0.], [0., 0., 0., 0.]], dtype=float32) raw.num_colors = 3 raw.raw_colors.shape = (3280, 4992) raw.raw_colors = array([[0, 1, 0, ..., 1, 0, 1], [3, 2, 3, ..., 2, 3, 2], [0, 1, 0, ..., 1, 0, 1], ..., [3, 2, 3, ..., 2, 3, 2], [0, 1, 0, ..., 1, 0, 1], [3, 2, 3, ..., 2, 3, 2]], dtype=uint8) raw.raw_colors_visible.shape = (3280, 4948) raw.raw_colors_visible = array([[0, 1, 0, ..., 1, 0, 1], [3, 2, 3, ..., 2, 3, 2], [0, 1, 0, ..., 1, 0, 1], ..., [3, 2, 3, ..., 2, 3, 2], [0, 1, 0, ..., 1, 0, 1], [3, 2, 3, ..., 2, 3, 2]], dtype=uint8) raw.raw_image_visible.shape = (3280, 4948) raw.raw_image_visible.dtype = dtype('uint16') raw.raw_image_visible = array([[ 864, 930, 854, ..., 647, 640, 658], [1062, 308, 977, ..., 219, 746, 219], [ 802, 919, 867, ..., 710, 625, 685], ..., [ 815, 322, 833, ..., 146, 511, 192], [ 769, 726, 751, ..., 513, 466, 468], [ 771, 321, 813, ..., 149, 525, 189]], dtype=uint16) raw.raw_image.min() = 0 raw.raw_image.max() = 4650 raw.raw_image.shape = (3280, 4992) raw.raw_image = array([[ 864, 930, 854, ..., 0, 0, 0], [1062, 308, 977, ..., 0, 0, 0], [ 802, 919, 867, ..., 0, 0, 0], ..., [ 815, 322, 833, ..., 0, 0, 0], [ 769, 726, 751, ..., 0, 0, 0], [ 771, 321, 813, ..., 0, 0, 0]], dtype=uint16) raw.raw_pattern = array([[0, 1], [3, 2]], dtype=uint8) raw.raw_type = <RawType.Flat: 0> raw.rgb_xyz_matrix = array([[ 0.8198, -0.2239, -0.0724], [-0.4871, 1.2389, 0.2798], [-0.1043, 0.205 , 0.7181], [ 0. , 0. , 0. ]], dtype=float32) raw.sizes = ImageSizes(raw_height=3280, raw_width=4992, height=3280, width=4948, top_margin=0, left_margin=0, iheight=3280, iwidth=4948, pixel_aspect=1.0, flip=0) δy = 0 δx = 0 ``` Why different `raw_width` and `width`? `raw_image` seems to have `0` in this case at the right side while it does not in `raw_image_visible`. Could verify color correctness in the RAW array. As in internship offer display image without interpolation would ease doing so it seems, in addition to be interesting. ```py help(rawpy.RawPy.postprocess) help(rawpy.Params) help(rawpy.DemosaicAlgorithm) ``` seem to show that we have to do it on our own. How to compute PRNU on these arrays? Only considering a given color channel seems to make sense. Pay attention that green might have a different shape. More precisely it does not seem to have the grid shape as the 2 other channels. ```py (4992 - 4948) / 2 ``` ``` 22.0 ``` ```py import math math.gcd(3280, 4992) math.gcd(3280, 4948) ``` ``` 16 4 ```

Benjamin_Loison commented 2024-04-15 15:04:49 +02:00

Author

Owner

Copy Link

Copy Source

Raw script:

import rawpy
import numpy as np
from tqdm import tqdm
import matplotlib.pyplot as plt

filename = 'r01b6aed6t.NEF'

colors = {
    'R': (255, 0, 0),
    'G': (0, 255, 0),
    'B': (0, 0, 255),
}

with rawpy.imread(filename) as raw:
    colorDesc = raw.color_desc.decode('ascii')
    colorsIndexes = [colors[color] for color in colorDesc]
    
    raw_colors_visible = raw.raw_colors_visible.copy()
    raw_image_visible = raw.raw_image_visible.copy()
    
    shape = raw_image_visible.shape
    # Consider the maximum value per channel does not help, as they all reach the same maximum value, at least on `r01b6aed6t.NEF`.
    rawImageVisibleMax = raw_image_visible.max()
    image = np.empty((shape[0], shape[1], 3), dtype=np.uint8)
    for y in tqdm(range(shape[0])):
        for x in range(shape[1]):
            value = np.array(colorsIndexes[raw_colors_visible[y][x]]) * raw_image_visible[y][x] / rawImageVisibleMax
            image[y][x] = value

assert colorDesc == 'RGBG'
assert np.array_equal(raw.raw_pattern, np.array([[3, 2], [0, 1]], dtype = np.uint8))

colorImages = {
    'R': image[1::2, ::2],
    'B': image[::2, 1::2],
}

colorImages['G'] = np.array([line[(0 if lineIndex % 2 == 0 else 1)::2] for lineIndex, line in enumerate(image)])

plt.imsave(f'{filename}.png', image)
for color, colorImage in colorImages.items():
    plt.imsave(f'{filename}_{color}.png', colorImage)

Raw script: ```py import rawpy import numpy as np from tqdm import tqdm import matplotlib.pyplot as plt filename = 'r01b6aed6t.NEF' colors = { 'R': (255, 0, 0), 'G': (0, 255, 0), 'B': (0, 0, 255), } with rawpy.imread(filename) as raw: colorDesc = raw.color_desc.decode('ascii') colorsIndexes = [colors[color] for color in colorDesc] raw_colors_visible = raw.raw_colors_visible.copy() raw_image_visible = raw.raw_image_visible.copy() shape = raw_image_visible.shape # Consider the maximum value per channel does not help, as they all reach the same maximum value, at least on `r01b6aed6t.NEF`. rawImageVisibleMax = raw_image_visible.max() image = np.empty((shape[0], shape[1], 3), dtype=np.uint8) for y in tqdm(range(shape[0])): for x in range(shape[1]): value = np.array(colorsIndexes[raw_colors_visible[y][x]]) * raw_image_visible[y][x] / rawImageVisibleMax image[y][x] = value assert colorDesc == 'RGBG' assert np.array_equal(raw.raw_pattern, np.array([[3, 2], [0, 1]], dtype = np.uint8)) colorImages = { 'R': image[1::2, ::2], 'B': image[::2, 1::2], } colorImages['G'] = np.array([line[(0 if lineIndex % 2 == 0 else 1)::2] for lineIndex, line in enumerate(image)]) plt.imsave(f'{filename}.png', image) for color, colorImage in colorImages.items(): plt.imsave(f'{filename}_{color}.png', colorImage) ```

Benjamin_Loison commented 2024-04-15 15:30:51 +02:00

Author

Owner

Copy Link

Copy Source

np.unique(raw_image_visible) shows that values 1, 2 etc are also taken, there is no gap.

`np.unique(raw_image_visible)` shows that values `1`, `2` etc are also taken, there is no gap.

Benjamin_Loison commented 2024-04-15 15:44:05 +02:00

Author

Owner

Copy Link

Copy Source

Taken with Nikon D90:

The bottom left white part is interesting to see the colored tiles in the three color channels.

At the top image border and mainly at left there are 2 red colored dots.

Red colored artificial flowers on the plates are also interesting to see almost only red tiles.

Increasing brightness and contrast also show the tiles everywhere.

Except if the Bayer filter is particularly involved in the PRNU, the color channel should not matter a lot, see #50. Related to #44.

Taken with `Nikon D90`:   The bottom left white part is interesting to see the colored tiles in the three color channels. At the top image border and mainly at left there are 2 red colored dots. Red colored artificial flowers on the plates are also interesting to see almost only red tiles. Increasing brightness and contrast also show the tiles everywhere. Except if the Bayer filter is particularly involved in the PRNU, the color channel should not matter a lot, see #50. Related to #44.

r01b6aed6t.png

23 MiB

r01b6aed6t.NEF.png

10 MiB

Benjamin_Loison referenced this issue 2024-04-15 16:17:43 +02:00

Estimate PRNU on easy real images #29

Benjamin_Loison commented 2024-04-15 20:04:58 +02:00

Author

Owner

Copy Link

Copy Source

  

r01b6aed6t.NEF_B.png

2.0 MiB

r01b6aed6t.NEF_R.png

2.0 MiB

r01b6aed6t.NEF_G.png

4.3 MiB

Benjamin_Loison commented 2024-04-15 20:34:52 +02:00

Author

Owner

Copy Link

Copy Source

Let us give a try with red as above red image looks easier to understand than the blue one.

Considering a subset of the green image seems appropriate as well assuming the independence of sensors which is maybe not exactly the case, especially for green ones.

Let us give a try with red as above [red image](https://gitea.lemnoslife.com/Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones/attachments/bb9f9619-de9f-472d-a93b-8d00fb02dae8) looks easier to understand than [the blue one](https://gitea.lemnoslife.com/Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones/attachments/b12256e5-080b-4e79-bbea-7931266e26d4). Considering a subset of the green image seems appropriate as well assuming the independence of sensors which is maybe not exactly the case, especially for green ones.

Benjamin_Loison commented 2024-04-15 21:38:21 +02:00

Author

Owner

Copy Link

Copy Source

Can verify the consistency of the subset of pixels by verifying areas using as most as possible a single color channel.

search_green.py:

#!/usr/bin/python3

from PIL import Image
import numpy as np
from tqdm import tqdm
import os

GREEN = np.array([0, 255, 0])

minimalDistance = None
minimalDistanceLocation = None

folder = 'raise_nef_png'

for file in tqdm(os.listdir(folder), 'Files'):
    with Image.open(f'{folder}/{file}') as image:
        imageNpArray = np.array(image)
        imageNpArrayShape = imageNpArray.shape
        for y in tqdm(range(imageNpArrayShape[0]), file):
            for x in range(imageNpArrayShape[1]):
                value = imageNpArray[y][x]
                distance = np.linalg.norm(value - GREEN)
                if minimalDistance is None or distance < minimalDistance:
                    print(file, minimalDistance, minimalDistanceLocation)
                    minimalDistance = distance
                    minimalDistanceLocation = (y, x)

ccd17f6bt.png 57.14017850864661 (1243, 3526)
...
r1a9acd94t.png: 100%|███████████████████████████████████████████████████████████| 4310/4310 [00:42<00:00, 100.42it/s]
Files:   2%|█▍                                                              | 175/8076 [2:35:00<113:59:13, 51.94s/it
...

Could iterate on images to find the most clear one.

Pay attention to possible visible offset.

Can verify the consistency of the subset of pixels by verifying areas using as most as possible a single color channel. `search_green.py`: ```py #!/usr/bin/python3 from PIL import Image import numpy as np from tqdm import tqdm import os GREEN = np.array([0, 255, 0]) minimalDistance = None minimalDistanceLocation = None folder = 'raise_nef_png' for file in tqdm(os.listdir(folder), 'Files'): with Image.open(f'{folder}/{file}') as image: imageNpArray = np.array(image) imageNpArrayShape = imageNpArray.shape for y in tqdm(range(imageNpArrayShape[0]), file): for x in range(imageNpArrayShape[1]): value = imageNpArray[y][x] distance = np.linalg.norm(value - GREEN) if minimalDistance is None or distance < minimalDistance: print(file, minimalDistance, minimalDistanceLocation) minimalDistance = distance minimalDistanceLocation = (y, x) ``` ``` ccd17f6bt.png 57.14017850864661 (1243, 3526) ... r1a9acd94t.png: 100%|███████████████████████████████████████████████████████████| 4310/4310 [00:42<00:00, 100.42it/s] Files: 2%|█▍ | 175/8076 [2:35:00<113:59:13, 51.94s/it ... ``` Could iterate on images to find the most clear one. Pay attention to possible `visible` offset.

Benjamin_Loison commented 2024-04-15 23:40:50 +02:00

Author

Owner

Copy Link

Copy Source

Related to Benjamin_Loison/rawpy/issues/1.

Related to [Benjamin_Loison/rawpy/issues/1](https://codeberg.org/Benjamin_Loison/rawpy/issues/1).

Benjamin_Loison commented 2024-04-16 00:42:10 +02:00

Author

Owner

Copy Link

Copy Source

help(rawpy.RawPy.raw_pattern)

Help on getset descriptor rawpy._rawpy.RawPy.raw_pattern:

raw_pattern
    The smallest possible Bayer pattern of this image.
    
    :rtype: ndarray, or None if not a flat RAW image

help(rawpy.RawPy.raw_colors_visible)

Help on getset descriptor rawpy._rawpy.RawPy.raw_colors_visible:

raw_colors_visible
    Like raw_colors but without margin.
    
    :rtype: ndarray of shape (hv,wv)

help(rawpy.RawPy.raw_colors)

Help on getset descriptor rawpy._rawpy.RawPy.raw_colors:

raw_colors
    An array of color indices for each pixel in the RAW image.
    Equivalent to calling raw_color(y,x) for each pixel.
    Only usable for flat RAW images (see raw_type property).
    
    :rtype: ndarray of shape (h,w)

help(rawpy.RawPy.raw_color)

Help on cython_function_or_method in module rawpy._rawpy:

raw_color(self, row, column)
    RawPy.raw_color(self, int row, int column) -> int
    
    Return color index for the given coordinates relative to the full RAW size.
    Only usable for flat RAW images (see raw_type property).

help(rawpy.RawPy.color_desc)

Help on getset descriptor rawpy._rawpy.RawPy.color_desc:

color_desc
    String description of colors numbered from 0 to 3 (RGBG,RGBE,GMCY, or GBTG).
    Note that same letters may not refer strictly to the same color.
    There are cameras with two different greens for example.

help(rawpy.RawPy.color_matrix)

Help on getset descriptor rawpy._rawpy.RawPy.color_matrix:

color_matrix
    Color matrix, read from file for some cameras, calculated for others. 
    
    :rtype: ndarray of shape (3,4)

```py help(rawpy.RawPy.raw_pattern) ``` ``` Help on getset descriptor rawpy._rawpy.RawPy.raw_pattern: raw_pattern The smallest possible Bayer pattern of this image. :rtype: ndarray, or None if not a flat RAW image ``` ```py help(rawpy.RawPy.raw_colors_visible) ``` ``` Help on getset descriptor rawpy._rawpy.RawPy.raw_colors_visible: raw_colors_visible Like raw_colors but without margin. :rtype: ndarray of shape (hv,wv) ``` ```py help(rawpy.RawPy.raw_colors) ``` ``` Help on getset descriptor rawpy._rawpy.RawPy.raw_colors: raw_colors An array of color indices for each pixel in the RAW image. Equivalent to calling raw_color(y,x) for each pixel. Only usable for flat RAW images (see raw_type property). :rtype: ndarray of shape (h,w) ``` ```py help(rawpy.RawPy.raw_color) ``` ``` Help on cython_function_or_method in module rawpy._rawpy: raw_color(self, row, column) RawPy.raw_color(self, int row, int column) -> int Return color index for the given coordinates relative to the full RAW size. Only usable for flat RAW images (see raw_type property). ``` ```py help(rawpy.RawPy.color_desc) ``` ``` Help on getset descriptor rawpy._rawpy.RawPy.color_desc: color_desc String description of colors numbered from 0 to 3 (RGBG,RGBE,GMCY, or GBTG). Note that same letters may not refer strictly to the same color. There are cameras with two different greens for example. ``` ```py help(rawpy.RawPy.color_matrix) ``` ``` Help on getset descriptor rawpy._rawpy.RawPy.color_matrix: color_matrix Color matrix, read from file for some cameras, calculated for others. :rtype: ndarray of shape (3,4) ```

Benjamin_Loison commented 2024-04-16 00:48:50 +02:00

Author

Owner

Copy Link

Copy Source

Related to Benjamin_Loison/vpv/issues/3 and Benjamin_Loison/darktable/issues/3.

Related to [Benjamin_Loison/vpv/issues/3](https://gitea.lemnoslife.com/Benjamin_Loison/vpv/issues/3) and [Benjamin_Loison/darktable/issues/3](https://codeberg.org/Benjamin_Loison/darktable/issues/3).

Benjamin_Loison referenced this issue from Benjamin_Loison/vpv 2024-04-16 01:00:27 +02:00

How to open whole `.nef`? #3

Benjamin_Loison commented 2024-04-16 01:11:51 +02:00

Author

Owner

Copy Link

Copy Source

Would be nice to not have to assert raw.raw_pattern.

Would be nice to not have to `assert` `raw.raw_pattern`.

Benjamin_Loison commented 2024-04-16 01:14:29 +02:00

Author

Owner

Copy Link

Copy Source

ra2c888f8t was done with Nikon D7000.

`ra2c888f8t` was done with Nikon D7000.

Benjamin_Loison commented 2024-04-16 01:16:20 +02:00

Author

Owner

Copy Link

Copy Source

import rawpy
import numpy as np
from tqdm import tqdm
import matplotlib.pyplot as plt

filename = 'ra2c888f8t.NEF'

colors = {
    'R': (255, 0, 0),
    'G': (0, 255, 0),
    'B': (0, 0, 255),
}

with rawpy.imread(filename) as raw:
    colorDesc = raw.color_desc.decode('ascii')
    assert colorDesc == 'RGBG'
    assert np.array_equal(raw.raw_pattern, np.array([[0, 1], [3, 2]], dtype = np.uint8))
    colorsIndexes = [colors[color] for color in colorDesc]
    
    raw_colors_visible = raw.raw_colors_visible.copy()
    raw_image_visible = raw.raw_image_visible.copy()
    
    shape = raw_image_visible.shape
    # Consider the maximum value per channel does not help, as they all reach the same maximum value, at least on `r01b6aed6t.NEF`.
    rawImageVisibleMax = raw_image_visible.max()
    image = np.empty((shape[0], shape[1], 3), dtype=np.uint8)
    for y in tqdm(range(shape[0])):
        for x in range(shape[1]):
            value = np.array(colorsIndexes[raw_colors_visible[y][x]]) * raw_image_visible[y][x] / rawImageVisibleMax
            image[y][x] = value

colorImages = {
    'R': image[::2, ::2],
    'G_S': image[1::2, ::2],
    'B': image[1::2, 1::2],
}

colorImages['G'] = np.array([line[(0 if lineIndex % 2 == 1 else 1)::2] for lineIndex, line in enumerate(image)])

plt.imsave(f'{filename}.png', image)
for color, colorImage in colorImages.items():
    plt.imsave(f'{filename}_{color}.png', colorImage)

```py import rawpy import numpy as np from tqdm import tqdm import matplotlib.pyplot as plt filename = 'ra2c888f8t.NEF' colors = { 'R': (255, 0, 0), 'G': (0, 255, 0), 'B': (0, 0, 255), } with rawpy.imread(filename) as raw: colorDesc = raw.color_desc.decode('ascii') assert colorDesc == 'RGBG' assert np.array_equal(raw.raw_pattern, np.array([[0, 1], [3, 2]], dtype = np.uint8)) colorsIndexes = [colors[color] for color in colorDesc] raw_colors_visible = raw.raw_colors_visible.copy() raw_image_visible = raw.raw_image_visible.copy() shape = raw_image_visible.shape # Consider the maximum value per channel does not help, as they all reach the same maximum value, at least on `r01b6aed6t.NEF`. rawImageVisibleMax = raw_image_visible.max() image = np.empty((shape[0], shape[1], 3), dtype=np.uint8) for y in tqdm(range(shape[0])): for x in range(shape[1]): value = np.array(colorsIndexes[raw_colors_visible[y][x]]) * raw_image_visible[y][x] / rawImageVisibleMax image[y][x] = value colorImages = { 'R': image[::2, ::2], 'G_S': image[1::2, ::2], 'B': image[1::2, 1::2], } colorImages['G'] = np.array([line[(0 if lineIndex % 2 == 1 else 1)::2] for lineIndex, line in enumerate(image)]) plt.imsave(f'{filename}.png', image) for color, colorImage in colorImages.items(): plt.imsave(f'{filename}_{color}.png', colorImage) ```

Benjamin_Loison pinned this 2024-04-16 01:39:56 +02:00

Benjamin_Loison commented 2024-04-16 01:40:15 +02:00

Author

Owner

Copy Link

Copy Source

Could consider the channel having the most variance on the considered set of images.

Could consider the channel having the most variance on the considered set of images.

Benjamin_Loison commented 2024-04-16 02:52:55 +02:00

Author

Owner

Copy Link

Copy Source

Are raw values bounded? Should check other rawpy.RawPy elements and compute statistics on a given image.

Are raw values bounded? Should check other `rawpy.RawPy` elements and compute statistics on a given image.

Benjamin_Loison commented 2024-04-16 03:02:13 +02:00

Author

Owner

Copy Link

Copy Source

With wavelet denoiser:

With bilateral denoiser:

Computing extremes of images: 100%|████████████████████████████████████████████████| 100/100 [00:32<00:00,  3.05it/s]
Denoising images:   8%|████▋                                                      | 8/100 [36:40<7:01:44, 275.05s/it]Denoising images: 100%|█████████████████████████████████████████████████████████| 100/100 [7:38:12<00:00, 274.93s/it]

With wavelet denoiser:   With bilateral denoiser: ``` Computing extremes of images: 100%|████████████████████████████████████████████████| 100/100 [00:32<00:00, 3.05it/s] Denoising images: 8%|████▋ | 8/100 [36:40<7:01:44, 275.05s/it]Denoising images: 100%|█████████████████████████████████████████████████████████| 100/100 [7:38:12<00:00, 274.93s/it] ```  

image.png

1.4 MiB

np_array.png

7.9 MiB

np_array.png

5.9 MiB

np_array_1.png

1.0 MiB

Benjamin_Loison referenced this issue from a commit 2024-04-16 03:02:59 +02:00

#49: Get PRNU by using RAW images

Benjamin_Loison commented 2024-04-16 03:10:52 +02:00

Author

Owner

Copy Link

Copy Source

Contrairement aux canaux rouge et bleu, l’utilisation du canal vert pour l’estimation du PRNU semble incertaine à cause de la répartition du vert. Est-ce qu’un décalage dans une des deux directions a vraiment une influence significative ?

> Contrairement aux canaux rouge et bleu, l’utilisation du canal vert pour l’estimation du PRNU semble incertaine à cause de la répartition du vert. Est-ce qu’un décalage dans une des deux directions a vraiment une influence significative ?

Benjamin_Loison commented 2024-04-16 03:38:11 +02:00

Author

Owner

Copy Link

Copy Source

Rafael ARW sky wavelet:

Rafael ARW wall wavelet:

I do not seem to find any visual feature in both. Except maybe the latter with an horizontal line at the bottom and possibly a lens circle at the top and bottom right.

Rafael ARW sky bilateral:

Computing extremes of images: 100%|████████████████████████████████████████████████| 108/108 [00:17<00:00,  6.12it/s]
minColor=1080
maxColor=6556
Denoising images for color green_right: 100%|██████████████████████████████████| 108/108 [12:20:38<00:00, 411.47s/it]

Rafael ARW wall bilateral:

Rafael ARW sky wavelet:  Rafael ARW wall wavelet:   I do not seem to find any visual feature in both. Except maybe the latter with an horizontal line at the bottom and possibly a lens circle at the top and bottom right. Rafael ARW sky bilateral:   ``` Computing extremes of images: 100%|████████████████████████████████████████████████| 108/108 [00:17<00:00, 6.12it/s] minColor=1080 maxColor=6556 Denoising images for color green_right: 100%|██████████████████████████████████| 108/108 [12:20:38<00:00, 411.47s/it] ``` Rafael ARW wall bilateral:  

np_array.png

13 MiB

image.png

11 MiB

image.png

800 KiB

image.png

10 MiB

image.png

2.1 MiB

mean_rafael_arw_bilateral_green_right.png

11 MiB

image.png

1.8 MiB

Benjamin_Loison commented 2024-04-16 04:25:44 +02:00

Author

Owner

Copy Link

Copy Source

Should implement estimating the PRNU for the 3 color channels. Considering a raw image as if not interpolating does not seem correct despite each color channel image having the same size as others, because the resulting pixel does not have the same light trajectory.

Just for visualizing can do so but otherwise should compute the distance as the sum for each color channel or regular euclidian for instance?

Should implement estimating the PRNU for the 3 color channels. Considering a raw image as if not interpolating does not seem correct despite each color channel image having the same size as others, because the resulting pixel does not have the same light trajectory. Just for visualizing can do so but otherwise should compute the distance as the sum for each color channel or regular euclidian for instance?

Benjamin_Loison commented 2024-04-16 19:30:04 +02:00

Author

Owner

Copy Link

Copy Source

As requested by Marina:

As requested by Marina:      

flat_001.NEF.png

15 MiB

flat_001.NEF_B.png

3.0 MiB

flat_001.NEF_G.png

6.7 MiB

flat_001.NEF_G_S.png

3.3 MiB

flat_001.NEF_R.png

3.4 MiB

flat_001.NEF

19 MiB

Benjamin_Loison commented 2024-04-18 00:14:53 +02:00

Author

Owner

Copy Link

Copy Source

Concerning merging multiple color channels PRNU estimations, I choose to input multiple images instead of multiple Numpy arrays, as I may adjust with GIMP to have brightness and contrast settings easing the visualization.

Il y aura probablement un problème d’échelles entre les canaux puisque deux pixels adjacents de couleurs différentes vont enregistrer des intensités potentiellement très différentes.

Flat-field wavelet:

Improved visualization thanks to brightness and contrast change, only contrast if I remember correctly:

Flat-field bilateral:

For unclear reasons the colors of green right and bottom differ quite significantly, displaying the actual distribution would help understand.

Concerning merging multiple color channels PRNU estimations, I choose to input multiple images instead of multiple Numpy arrays, as I may adjust with GIMP to have brightness and contrast settings easing the visualization. Il y aura probablement un problème d’échelles entre les canaux puisque deux pixels adjacents de couleurs différentes vont enregistrer des intensités potentiellement très différentes. Flat-field wavelet:  Improved visualization thanks to brightness and contrast change, only contrast if I remember correctly:  Flat-field bilateral:   For unclear reasons the colors of green right and bottom differ quite significantly, displaying the actual distribution would help understand.

image.png

3.8 MiB

multipleColors.png

33 MiB

multipleColors.png

23 MiB

multipleColors_brightness_contrast_modified.png

1.8 MiB

Benjamin_Loison referenced this issue 2024-04-18 00:15:21 +02:00

Automate GIMP best brightness and contrast to ease visualization? #52

Benjamin_Loison commented 2024-04-19 16:53:44 +02:00

Author

Owner

Copy Link

Copy Source

Correctly assembled 4 PRNUs:

Do not have lines anymore, as do not - npArrayMin, as I thought it was to make matplotlib happy but it does not seem to care and it only introduce bias by introducing a different offset per color channel.

Also note that we do not notice the Bayer filter pattern, i.e. red being particularly present as the wall look orange for instance, as we are considering noise for each color channel, so the resulting image is just about difference in each color channel which is more similar than absolute values of each color channel.

Improved brightness and contrast:

Correctly assembled 4 PRNUs:  Do not have lines anymore, as do not `- npArrayMin`, as I thought it was to make matplotlib happy but it does not seem to care and it only introduce bias by introducing a different offset per color channel. Also note that we do not notice the Bayer filter pattern, i.e. red being particularly present as the wall look orange for instance, as we are considering noise for each color channel, so the resulting image is just about difference in each color channel which is more similar than absolute values of each color channel. Improved brightness and contrast: 

final.png

18 MiB

image.png

4.8 MiB

Benjamin_Loison referenced this issue from a commit 2024-04-19 17:19:55 +02:00

#49: Remove shift per color channel

Benjamin_Loison referenced this issue from a commit 2024-04-19 17:21:58 +02:00

#49: Remove intermediary bias by not working on images but numpy arrays instead

Benjamin_Loison commented 2024-04-21 19:47:21 +02:00

Author

Owner

Copy Link

Copy Source

4 PRNUs bilateral denoiser assembled for Rafael wall:

Modified brightness and contrast:

Do we also notice lines with wavelet denoiser and on sky too?

Seems to have similar behavior with Rafael sky with mean denoiser (related to #57):

Modified brightness and contrast:

Similar result with the wall:

Could upload 4 PRNUs bilateral denoiser assembled for Rafael sky.

Why is the Bayer filter that visible?

First image histogram:

Mean of images histogram:

First Bayer filter occurrence histogram:

First Bayer filter occurrence minus mean of images histogram:

RAISE flat-field incorrect PRNU estimation with mean denoiser (the PRNU estimation is just the mean of images) with 4 color channels assembled:

With modified brightness and contrast:

The respective corrected images:

Annotated flat_099.tif showing some circles on the camera.

RAISE flat-field timeline:

Rafael 24/04/24:

3064 to 3078:

22/04/24:

23/04/24:

4 PRNUs bilateral denoiser assembled for Rafael wall:  Modified brightness and contrast:  Do we also notice lines with wavelet denoiser and on sky too? Seems to have similar behavior with Rafael sky with mean denoiser (related to #57):  Modified brightness and contrast:  Similar result with the wall:  Could upload 4 PRNUs bilateral denoiser assembled for Rafael sky. Why is the Bayer filter that visible? First image histogram:  Mean of images histogram:  First Bayer filter occurrence histogram:  First Bayer filter occurrence minus mean of images histogram:  RAISE flat-field incorrect PRNU estimation with mean denoiser (the PRNU estimation is just the mean of images) with 4 color channels assembled:  With modified brightness and contrast:  The respective corrected images:   Annotated `flat_099.tif` showing some circles on the camera.    RAISE flat-field timeline:  Rafael 24/04/24:  3064 to 3078:  22/04/24:  23/04/24: 

multiple_colors.png

30 MiB

image.png

3.3 MiB

image.png

980 KiB

mean_rafael_arw_sky_mean_multiple_colors.png

25 MiB

mean_rafael_arw_wall_mean_multiple_colors.png

37 MiB

first_image.svg

170 KiB

mean.svg

167 KiB

first_bayer_filter_occurrence.svg

64 KiB

first_bayer_filter_occurrence_minus_mean.svg

63 KiB

mean_flat-field_NEF_mean_multiple_colors.png

30 MiB

mean_flat-field_NEF_mean_multiple_colors_brightness_contrast_changed.png

2.8 MiB

flat_099_annotated.tif

46 MiB

center_wall_marker_locations.svg

41 KiB

distance_between_a_wall_marker_location_and_the_next_one.svg

24 KiB

raise_flat-field_timeline.svg

268 KiB

rafael_240424.svg

395 KiB

rafael_240424_3064_to_3078.svg

129 KiB

rafael_240424_220424.svg

204 KiB

rafael_240424_230424.png

277 KiB

mean_flat-field_NEF_mean_multiple_colors.png

6.0 MiB

image.png

6.1 MiB

Benjamin_Loison referenced this issue 2024-04-27 14:57:43 +02:00

Use mean as denoiser #57

Benjamin_Loison referenced this issue 2024-04-27 16:32:17 +02:00

Make `split_and_compare_prnus_of_subgroups.py` compatible with RAW images #59

Benjamin_Loison referenced this issue 2024-04-30 02:34:29 +02:00

How to deal with multiple color channels? #11

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

master

prnu_written_as_such

evaluation

Labels

Clear labels

bug

Something is not working

Context-Adaptive Interpolator

duplicate

This issue or pull request already exists

enhancement

New feature

epic

A task that is going to take more than a day to complete.

help wanted

Need some help

high priority

invalid

Something is wrong

left for future work

low priority

medium

A task that should take less than a day to complete.

medium priority

meta

question

More information is needed

quick

A task that should take less than two hours to complete.

wontfix

This won't be fixed

No labels enhancement medium medium priority

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

Benjamin_Loison (Benjamin Loison)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones#49