T
- the type being queuedpublic class QueueFileOffHeapDispatchQueue<T> extends Object implements DispatchQueue<T>
DispatchQueue
that first attempts to queue items in memory and upon overflowing the allocated in-memory
queue writes items "off heap" directly to disk via a file. The in-memory queue is volatile and if the process
crashes its contents are lost. The contents written to disk are durable and in the event of a crash will be reloaded.
This queue can be configured to only queue to memory by specifying the maximum off-heap size of 0. Using this
configuration causes enqueue(T, java.lang.String)
to block when the in-memory queue fills up rather than writing to the off-heap
queue.
Before queued items are written to disk they are first accumulated in a batch to limit the number of discrete writes we make to disk. The batched items are considered part of the in-memory portion of the queue and are also volatile.
DispatchQueue.EnqueueResult
Constructor and Description |
---|
QueueFileOffHeapDispatchQueue(java.util.function.Function<T,byte[]> serializer,
java.util.function.Function<byte[],T> deserializer,
String moduleName,
Path filePath,
int inMemoryQueueSize,
int batchSize,
long maxFileSizeInBytes) |
Modifier and Type | Method and Description |
---|---|
long |
checkFileSize() |
Map.Entry<String,T> |
dequeue()
On every call to dequeue, if the off-heap queue is configured, we check the file for an entry and drain it to the
in-memory queue provided there is room.
|
DispatchQueue.EnqueueResult |
enqueue(T message,
String key)
When enqueueing we prefer the in-memory queue unless the file based queue is already utilized.
|
int |
getSize() |
boolean |
isFull() |
public QueueFileOffHeapDispatchQueue(java.util.function.Function<T,byte[]> serializer, java.util.function.Function<byte[],T> deserializer, String moduleName, Path filePath, int inMemoryQueueSize, int batchSize, long maxFileSizeInBytes) throws IOException
IOException
public long checkFileSize()
public DispatchQueue.EnqueueResult enqueue(T message, String key) throws WriteFailedException
We only write to the file based queue when we have a full batch ready. The batch container is then emptied after being written to disk.
enqueue
in interface DispatchQueue<T>
WriteFailedException
public Map.Entry<String,T> dequeue() throws InterruptedException
After completely draining the queue on disk we check the existing batch for entries and drain them next.
dequeue
in interface DispatchQueue<T>
InterruptedException
- if interrupted while waitingpublic boolean isFull()
isFull
in interface DispatchQueue<T>
public int getSize()
getSize
in interface DispatchQueue<T>
Copyright © 2020. All rights reserved.