org.apache.hadoop.hive.llap.cache

Class BuddyAllocator

java.lang.Object

org.apache.hadoop.hive.llap.cache.BuddyAllocator

All Implemented Interfaces:

org.apache.hadoop.hive.common.io.Allocator, BuddyAllocatorMXBean, EvictionAwareAllocator, LlapOomDebugDump

public final class BuddyAllocator
extends Object
implements EvictionAwareAllocator, BuddyAllocatorMXBean, LlapOomDebugDump

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hive.common.io.Allocator

org.apache.hadoop.hive.common.io.Allocator.AllocatorOutOfMemoryException, org.apache.hadoop.hive.common.io.Allocator.BufferObjectFactory

Constructor Summary

Constructors

Constructor and Description
BuddyAllocator(boolean isDirectVal, boolean isMappedVal, int minAllocVal, int maxAllocVal, int arenaCount, long maxSizeVal, long defragHeadroom, String mapPath, MemoryManager memoryManager, LlapDaemonCacheMetrics metrics, String discardMethod)
BuddyAllocator(org.apache.hadoop.conf.Configuration conf, MemoryManager mm, LlapDaemonCacheMetrics metrics)

Method Summary

Modifier and Type	Method and Description
void	allocateMultiple(org.apache.hadoop.hive.common.io.encoded.MemoryBuffer[] dest, int size)
void	allocateMultiple(org.apache.hadoop.hive.common.io.encoded.MemoryBuffer[] dest, int size, org.apache.hadoop.hive.common.io.Allocator.BufferObjectFactory factory)
org.apache.hadoop.hive.common.io.encoded.MemoryBuffer	createUnallocated() Deprecated.
void	deallocate(org.apache.hadoop.hive.common.io.encoded.MemoryBuffer buffer)
void	deallocateEvicted(org.apache.hadoop.hive.common.io.encoded.MemoryBuffer buffer)
String	debugDumpForOom()
void	debugDumpShort(StringBuilder sb) Arbitrarily, we start getting the state from Allocator.
long	determineMaxMmSize(long defragHeadroom, long maxMmSize)
void	dumpTestLog()
int	getArenaSize() Gets the arena size.
boolean	getIsDirect() Gets if bytebuffers are allocated directly offheap.
int	getMaxAllocation() Gets maximum allocation size of allocator.
long	getMaxCacheSize() Gets the maximum cache size.
int	getMinAllocation() Gets minimum allocation size of allocator.
boolean	isDirectAlloc()
int	offsetFromHeaderIndex(int lastSplitNextHeader)
int	validateAndDetermineArenaSize(int arenaCount, long maxSizeVal)
long	validateAndDetermineMaxSize(long maxSizeVal)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BuddyAllocator

public BuddyAllocator(org.apache.hadoop.conf.Configuration conf,
                      MemoryManager mm,
                      LlapDaemonCacheMetrics metrics)

BuddyAllocator

public BuddyAllocator(boolean isDirectVal,
                      boolean isMappedVal,
                      int minAllocVal,
                      int maxAllocVal,
                      int arenaCount,
                      long maxSizeVal,
                      long defragHeadroom,
                      String mapPath,
                      MemoryManager memoryManager,
                      LlapDaemonCacheMetrics metrics,
                      String discardMethod)

Method Detail

determineMaxMmSize

public long determineMaxMmSize(long defragHeadroom,
                               long maxMmSize)

validateAndDetermineMaxSize

public long validateAndDetermineMaxSize(long maxSizeVal)

validateAndDetermineArenaSize

public int validateAndDetermineArenaSize(int arenaCount,
                                         long maxSizeVal)

allocateMultiple

public void allocateMultiple(org.apache.hadoop.hive.common.io.encoded.MemoryBuffer[] dest,
                             int size)
                      throws org.apache.hadoop.hive.common.io.Allocator.AllocatorOutOfMemoryException

Specified by:

allocateMultiple in interface org.apache.hadoop.hive.common.io.Allocator

Throws:

org.apache.hadoop.hive.common.io.Allocator.AllocatorOutOfMemoryException

allocateMultiple

public void allocateMultiple(org.apache.hadoop.hive.common.io.encoded.MemoryBuffer[] dest,
                             int size,
                             org.apache.hadoop.hive.common.io.Allocator.BufferObjectFactory factory)
                      throws org.apache.hadoop.hive.common.io.Allocator.AllocatorOutOfMemoryException

Specified by:

allocateMultiple in interface org.apache.hadoop.hive.common.io.Allocator

Throws:

org.apache.hadoop.hive.common.io.Allocator.AllocatorOutOfMemoryException

debugDumpShort

public void debugDumpShort(StringBuilder sb)

Arbitrarily, we start getting the state from Allocator. Allocator calls MM which calls the policies that call the eviction dispatcher that calls the caches. See init - these all are connected in a cycle, so we need to make sure the who-calls-whom order is definite.

Specified by:

debugDumpShort in interface LlapOomDebugDump

deallocate

public void deallocate(org.apache.hadoop.hive.common.io.encoded.MemoryBuffer buffer)

Specified by:

deallocate in interface org.apache.hadoop.hive.common.io.Allocator

deallocateEvicted

public void deallocateEvicted(org.apache.hadoop.hive.common.io.encoded.MemoryBuffer buffer)

Specified by:

deallocateEvicted in interface EvictionAwareAllocator

isDirectAlloc

public boolean isDirectAlloc()

Specified by:

isDirectAlloc in interface org.apache.hadoop.hive.common.io.Allocator

createUnallocated

@Deprecated
public org.apache.hadoop.hive.common.io.encoded.MemoryBuffer createUnallocated()

Deprecated.

Specified by:

createUnallocated in interface org.apache.hadoop.hive.common.io.Allocator

getIsDirect

public boolean getIsDirect()

Description copied from interface: BuddyAllocatorMXBean

Gets if bytebuffers are allocated directly offheap.

Specified by:

getIsDirect in interface BuddyAllocatorMXBean

Returns:

gets if direct bytebuffer allocation

getMinAllocation

public int getMinAllocation()

Description copied from interface: BuddyAllocatorMXBean

Gets minimum allocation size of allocator.

Specified by:

getMinAllocation in interface BuddyAllocatorMXBean

Returns:

minimum allocation size

getMaxAllocation

public int getMaxAllocation()

Description copied from interface: BuddyAllocatorMXBean

Gets maximum allocation size of allocator.

Specified by:

getMaxAllocation in interface org.apache.hadoop.hive.common.io.Allocator

Specified by:

getMaxAllocation in interface BuddyAllocatorMXBean

Returns:

maximum allocation size

getArenaSize

public int getArenaSize()

Description copied from interface: BuddyAllocatorMXBean

Gets the arena size.

Specified by:

getArenaSize in interface BuddyAllocatorMXBean

Returns:

arena size

getMaxCacheSize

public long getMaxCacheSize()

Description copied from interface: BuddyAllocatorMXBean

Gets the maximum cache size.

Specified by:

getMaxCacheSize in interface BuddyAllocatorMXBean

Returns:

max cache size

offsetFromHeaderIndex

public int offsetFromHeaderIndex(int lastSplitNextHeader)

debugDumpForOom

public String debugDumpForOom()

Specified by:

debugDumpForOom in interface LlapOomDebugDump

dumpTestLog

public void dumpTestLog()