Benjamin_Loison/Robust_image_source_identification_on_modern_smartphones
1
0
Fork 0
Code Issues 68 Pull Requests Packages Projects Releases Wiki Activity

#62: First lazy load try

Browse Source 
```
OPERATION = <Operation.LOAD_NPY: 3>
RESOLUTION = 100
Image:   0%|▍                                                                                            | 1/200 [00:00<00:00, 5084.00it/s]
Camera:   0%|                                                                                                        | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/benjamin/robust_image_source_identification_on_modern_smartphones/datasets/raise/./benchmark_load_part_of_images.py", line 43, in <module>
    rawColorChannels = np.load(numpyFilePath, mmap_mode = 'r', allow_pickle = True)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benjamin/venv/lib/python3.11/site-packages/numpy/lib/npyio.py", line 453, in load
    return format.open_memmap(file, mode=mmap_mode,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benjamin/venv/lib/python3.11/site-packages/numpy/lib/format.py", line 932, in open_memmap
    raise ValueError(msg)
ValueError: Array can't be memory-mapped: Python objects in dtype.
```
This commit is contained in:
Benjamin Loison 2024-05-13 18:30:28 +02:00
parent ac32abede1
commit f85e63b6e0
No known key found for this signature in database
1 changed files with 6 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
datasets/raise/benchmark_load_part_of_images.py
View File

@ -16,7 +16,10 @@ class Operation(Enum):
SAVE = auto()
LOAD_NPY = auto()
OPERATION = Operation.LOAD_RAW
def __str__(self):
return self.name
OPERATION = Operation.LOAD_NPY
RESOLUTION = 100
print(f'{OPERATION = }')
if OPERATION == Operation.LOAD_NPY:
@ -37,7 +40,7 @@ for camera in tqdm(IMAGES_CAMERAS_FOLDER, 'Camera'):
if OPERATION == Operation.SAVE:
np.save(numpyFilePath, rawColorChannels)
if OPERATION == Operation.LOAD_NPY:
rawColorChannels = np.load(numpyFilePath, allow_pickle = True)
rawColorChannels = np.load(numpyFilePath, mmap_mode = 'r', allow_pickle = True)
rawColorChannelsItem = rawColorChannels.item()
for color in Color:
print(color, rawColorChannelsItem[color].mean())
print(color, rawColorChannelsItem[color][:RESOLUTION].mean())