5

我正在尝试确定任何给定的 TYPE_INT_ARGB 将使用多少堆BufferedImage,以便对于正在执行一些图像处理的程序,我可以根据我们提供的图像大小设置一个合理的最大堆。

我编写了以下程序作为测试,然后我用它来确定它在没有 的情况下运行的最小最大堆OutOfMemoryError

import java.awt.image.BufferedImage;

public class Test {
  public static void main(String[] args) {
    final int w = Integer.parseInt(args[0]);
    final int h = Integer.parseInt(args[1]);

    final BufferedImage img =
      new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB);

    System.out.println((4*w*h) >> 20);
  }
}

(打印的值是存储像素数据的预期大小int[]BufferedImage)我期望找到的是所需的最大堆类似于x + c,其中x是数据数组的大小,并且c是一个常数加载的类、BufferedImage对象等的大小。这是我发现的(所有值都以 MB 为单位):

4*w*h 最小最大堆
----- ------------
  5 -
 10 15
 20 31
 40 61
 80 121
160 241

1.5x非常适合观察。(请注意,我发现 5MB 图像没有最小值。)我不明白我所看到的。这些额外的字节是什么?

4

3 回答 3

4

在 1.6.0_16 和 1.6.0_20 之间的某处引入的 Oracle 虚拟机中似乎存在一个错误。您甚至可以将问题简化为分配一个 int 数组,因为该问题不仅与 BufferedImage 有关。

对于 1.6.0_16,我需要至少 413 MB 堆来分配一个包含 100,000,000 个元素的 int 数组,这似乎是合理的。在 1.6.0_20 中,同样的操作至少需要 573 MB 的堆空间,尽管在分配数组之后实际只使用了大约 400,000,000 字节。

于 2010-10-04T11:37:56.097 回答
1

在进一步调查中,问题似乎是堆中的老年代无法充分扩展以容纳图像的数据数组,尽管堆中有足够的空闲内存。

有关如何扩展老年代的更多详细信息,请参阅此问题

于 2010-11-24T17:38:58.663 回答
1

问题在于 BufferedImage 对象以未压缩的格式将图像存储在内存中。对此有一个有效的解决方案:您可以将图像存储在硬盘上,而不必担心堆大小或物理内存限制。它可以存储最大 2,147,483,647 像素(或 46,340 x 46,340 像素)。BigBufferedImage解决了这个问题。

创建一个空的 BigBufferedImage:

BigBufferedImage image = BigBufferedImage.create(
        tempDir, width, height, type);

将现有图像加载到 BigBufferedImage:

BigBufferedImage image = BigBufferedImage.create(
        imagePath, tempDir, type);

渲染图像的一部分:

part = image.getSubimage(x, y, width, height);

BigBufferedImage 的实现:

package com.pulispace.mc.ui.panorama.util;

/*
 * This class is part of MCFS (Mission Control - Flight Software) a development 
 * of Team Puli Space, official Google Lunar XPRIZE contestant. 
 * This class is released under Creative Commons CC0.
 * @author Zsolt Pocze, Dimitry Polivaev
 * Please like us on facebook, and/or join our Small Step Club.
 * http://www.pulispace.com
 * https://www.facebook.com/pulispace
 * http://nyomdmegteis.hu/en/
 */
import java.awt.Point;
import java.awt.Rectangle;
import java.awt.color.ColorSpace;
import java.awt.image.BandedSampleModel;
import java.awt.image.BufferedImage;
import java.awt.image.ColorModel;
import java.awt.image.ComponentColorModel;
import java.awt.image.DataBuffer;
import java.awt.image.Raster;
import java.awt.image.RenderedImage;
import java.awt.image.SampleModel;
import java.awt.image.WritableRaster;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.Hashtable;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.imageio.ImageIO;
import javax.imageio.ImageReadParam;
import javax.imageio.ImageReader;
import javax.imageio.stream.ImageInputStream;
import sun.nio.ch.DirectBuffer;

public class BigBufferedImage extends BufferedImage {

    private static final String TMP_DIR = System.getProperty("java.io.tmpdir");
    public static final int MAX_PIXELS_IN_MEMORY =  1024 * 1024;

    public static BufferedImage create(int width, int height, int imageType) {
        if (width * height > MAX_PIXELS_IN_MEMORY) {
            try {
                final File tempDir = new File(TMP_DIR);
                return createBigBufferedImage(tempDir, width, height, imageType);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        } else {
            return new BufferedImage(width, height, imageType);
        }
    }

    public static BufferedImage create(File inputFile, int imageType) throws IOException {
        try (ImageInputStream stream = ImageIO.createImageInputStream(inputFile);) {
            Iterator<ImageReader> readers = ImageIO.getImageReaders(stream);
            if (readers.hasNext()) {
                try {
                    ImageReader reader = readers.next();
                    reader.setInput(stream, true, true);
                    int width = reader.getWidth(reader.getMinIndex());
                    int height = reader.getHeight(reader.getMinIndex());
                    BufferedImage image = create(width, height, imageType);
                    int cores = Math.max(1, Runtime.getRuntime().availableProcessors() / 2);
                    int block = Math.min(MAX_PIXELS_IN_MEMORY / cores / width, (int) (Math.ceil(height / (double) cores)));
                    ExecutorService generalExecutor = Executors.newFixedThreadPool(cores);
                    List<Callable<ImagePartLoader>> partLoaders = new ArrayList<>();
                    for (int y = 0; y < height; y += block) {
                        partLoaders.add(new ImagePartLoader(
                            y, width, Math.min(block, height - y), inputFile, image));
                    }
                    generalExecutor.invokeAll(partLoaders);
                    generalExecutor.shutdown();
                    return image;
                } catch (InterruptedException ex) {
                    Logger.getLogger(BigBufferedImage.class.getName()).log(Level.SEVERE, null, ex);
                }
            }
        }
        return null;
    }

    private static BufferedImage createBigBufferedImage(File tempDir, int width, int height, int imageType)
        throws FileNotFoundException, IOException {
        FileDataBuffer buffer = new FileDataBuffer(tempDir, width * height, 4);
        ColorModel colorModel = null;
        BandedSampleModel sampleModel = null;
        switch (imageType) {
            case TYPE_INT_RGB:
                colorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB),
                    new int[]{8, 8, 8, 0},
                    false,
                    false,
                    ComponentColorModel.TRANSLUCENT,
                    DataBuffer.TYPE_BYTE);
                sampleModel = new BandedSampleModel(DataBuffer.TYPE_BYTE, width, height, 3);
                break;
            case TYPE_INT_ARGB:
                colorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB),
                    new int[]{8, 8, 8, 8},
                    true,
                    false,
                    ComponentColorModel.TRANSLUCENT,
                    DataBuffer.TYPE_BYTE);
                sampleModel = new BandedSampleModel(DataBuffer.TYPE_BYTE, width, height, 4);
                break;
            default:
                throw new IllegalArgumentException("Unsupported image type: " + imageType);
        }
        SimpleRaster raster = new SimpleRaster(sampleModel, buffer, new Point(0, 0));
        BigBufferedImage image = new BigBufferedImage(colorModel, raster, colorModel.isAlphaPremultiplied(), null);
        return image;
    }

    private static class ImagePartLoader implements Callable<ImagePartLoader> {

        private final int y;
        private final BufferedImage image;
        private final Rectangle region;
        private final File file;

        public ImagePartLoader(int y, int width, int height, File file, BufferedImage image) {
            this.y = y;
            this.image = image;
            this.file = file;
            region = new Rectangle(0, y, width, height);
        }

        @Override
        public ImagePartLoader call() throws Exception {
            Thread.currentThread().setPriority((Thread.MIN_PRIORITY + Thread.NORM_PRIORITY) / 2);
            try (ImageInputStream stream = ImageIO.createImageInputStream(file);) {
                Iterator<ImageReader> readers = ImageIO.getImageReaders(stream);
                if (readers.hasNext()) {
                    ImageReader reader = readers.next();
                    reader.setInput(stream, true, true);
                    ImageReadParam param = reader.getDefaultReadParam();
                    param.setSourceRegion(region);
                    BufferedImage part = reader.read(0, param);
                    Raster source = part.getRaster();
                    WritableRaster target = image.getRaster();
                    target.setRect(0, y, source);
                }
            }
            return ImagePartLoader.this;
        }
    }

    private BigBufferedImage(ColorModel cm, SimpleRaster raster, boolean isRasterPremultiplied, Hashtable<?, ?> properties) {
        super(cm, raster, isRasterPremultiplied, properties);
    }

    public void dispose() {
        ((SimpleRaster) getRaster()).dispose();
    }

    public static void dispose(RenderedImage image) {
        if (image instanceof BigBufferedImage) {
            ((BigBufferedImage) image).dispose();
        }
    }

    private static class SimpleRaster extends WritableRaster {

        public SimpleRaster(SampleModel sampleModel, FileDataBuffer dataBuffer, Point origin) {
            super(sampleModel, dataBuffer, origin);
        }

        public void dispose() {
            ((FileDataBuffer) getDataBuffer()).dispose();
        }

    }

    private static final class FileDataBufferDeleterHook extends Thread {

        static {
            Runtime.getRuntime().addShutdownHook(new FileDataBufferDeleterHook());
        }

        private static final HashSet<FileDataBuffer> undisposedBuffers = new HashSet<>();

        @Override
        public void run() {
            final FileDataBuffer[] buffers = undisposedBuffers.toArray(new FileDataBuffer[0]);
            for (FileDataBuffer b : buffers) {
                b.disposeNow();
            }
        }
    }

    private static class FileDataBuffer extends DataBuffer {

        private final String id = "buffer-" + System.currentTimeMillis() + "-" + ((int) (Math.random() * 1000));
        private File dir;
        private String path;
        private File[] files;
        private RandomAccessFile[] accessFiles;
        private MappedByteBuffer[] buffer;

        public FileDataBuffer(File dir, int size) throws FileNotFoundException, IOException {
            super(TYPE_BYTE, size);
            this.dir = dir;
            init();
        }

        public FileDataBuffer(File dir, int size, int numBanks) throws FileNotFoundException, IOException {
            super(TYPE_BYTE, size, numBanks);
            this.dir = dir;
            init();
        }

        private void init() throws FileNotFoundException, IOException {
            FileDataBufferDeleterHook.undisposedBuffers.add(this);
            if (dir == null) {
                dir = new File(".");
            }
            if (!dir.exists()) {
                throw new RuntimeException("FileDataBuffer constructor parameter dir does not exist: " + dir);
            }
            if (!dir.isDirectory()) {
                throw new RuntimeException("FileDataBuffer constructor parameter dir is not a directory: " + dir);
            }
            path = dir.getPath() + "/" + id;
            File subDir = new File(path);
            subDir.mkdir();
            buffer = new MappedByteBuffer[banks];
            accessFiles = new RandomAccessFile[banks];
            files = new File[banks];
            for (int i = 0; i < banks; i++) {
                File file = files[i] = new File(path + "/bank" + i + ".dat");
                final RandomAccessFile randomAccessFile = accessFiles[i] = new RandomAccessFile(file, "rw");
                buffer[i] = randomAccessFile.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, getSize());
            }
        }

        @Override
        public int getElem(int bank, int i) {
            return buffer[bank].get(i) & 0xff;
        }

        @Override
        public void setElem(int bank, int i, int val) {
            buffer[bank].put(i, (byte) val);
        }

        @Override
        protected void finalize() throws Throwable {
            dispose();
        }

        private void disposeNow() {
            final MappedByteBuffer[] disposedBuffer = this.buffer;
            this.buffer = null;
            disposeNow(disposedBuffer);
        }

        public void dispose() {
            final MappedByteBuffer[] disposedBuffer = this.buffer;
            this.buffer = null;
            new Thread() {
                @Override
                public void run() {
                    disposeNow(disposedBuffer);
                }
            }.start();
        }

        private void disposeNow(final MappedByteBuffer[] disposedBuffer) {
            FileDataBufferDeleterHook.undisposedBuffers.remove(this);
            if (disposedBuffer != null) {
                for (MappedByteBuffer b : disposedBuffer) {
                    ((DirectBuffer) b).cleaner().clean();
                }
            }
            if (accessFiles != null) {
                for (RandomAccessFile file : accessFiles) {
                    try {
                        file.close();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
                accessFiles = null;
            }
            if (files != null) {
                for (File file : files) {
                    file.delete();
                }
                files = null;
            }
            if (path != null) {
                new File(path).delete();
                path = null;
            }
        }

    }
}
于 2018-11-08T10:15:58.100 回答