AzureDLFile Class

Open ADL key as a file. Data is only loaded and cached on demand.

Inheritance

builtins.object

AzureDLFile

Constructor

AzureDLFile(azure, path, mode='rb', blocksize=33554432, delimiter=None)

Parameters

Name	Description
azure Required	<xref:<xref:azure.datalake.store.core.azure connection>>
path Required	AzureDLPath location of file
mode	Default value: rb
blocksize	int Size of the write or read-ahead buffer. For writing(and appending, will be truncated to 4MB (2**22). Default value: 33554432
delimiter	bytes or None If specified and in write mode, each flush will send data terminating on this bytestring, potentially leaving some data in the buffer. Default value: None

Examples

>>> adl = AzureDLFileSystem()  
>>> with adl.open('my-dir/my-file.txt', mode='rb') as f:  
...     f.read(10)  

Methods

close	Close file If in write mode, causes flush of any unwritten data.
flush	Write buffered data to ADL. Without delimiter: Uploads the current buffer. With delimiter: writes an amount of data less than or equal to the block-size, which ends on the delimiter, until buffer is smaller than the blocksize. If there is no delimiter in a block uploads whole block. If force=True, flushes all data in the buffer, even if it doesn't end with a delimiter; appropriate when closing the file.
info	File information about this path
next
read	Return data from cache, or fetch pieces as necessary
read1	Return data from cache, or fetch pieces as necessary
readable	Return whether the AzureDLFile was opened for reading
readinto	Reads data into buffer b
readline	Read and return a line from the stream. If length is specified, at most size bytes will be read.
readlines	Return all lines in a file as a list
seek	Set current file location
seekable	Return whether the AzureDLFile is seekable (only in read mode)
tell	Current file location
writable	Return whether the AzureDLFile was opened for writing
write	Write data to buffer. Buffer only sent to ADL on flush() or if buffer is bigger than blocksize.

close

Close file

If in write mode, causes flush of any unwritten data.

close()

flush

Write buffered data to ADL.

Without delimiter: Uploads the current buffer.

With delimiter: writes an amount of data less than or equal to the block-size, which ends on the delimiter, until buffer is smaller than the blocksize. If there is no delimiter in a block uploads whole block.

If force=True, flushes all data in the buffer, even if it doesn't end with a delimiter; appropriate when closing the file.

flush(syncFlag='METADATA', force=False)

Parameters

Name	Description
syncFlag	Default value: METADATA
force	Default value: False

info

File information about this path

info()

next

next()

read

Return data from cache, or fetch pieces as necessary

read(length=-1)

Parameters

Name	Description
length	int(<xref:-1>) Number of bytes to read; if <0, all remaining bytes. Default value: -1

read1

Return data from cache, or fetch pieces as necessary

read1(length=-1)

Parameters

Name	Description
length	int(<xref:-1>) Number of bytes to read; if <0, all remaining bytes. Default value: -1

readable

Return whether the AzureDLFile was opened for reading

readable()

readinto

Reads data into buffer b

readinto(b)

Parameters

Name	Description
b Required	bytearray Buffer to which bytes are read into

Returns

Type	Description
<xref:Returns> <xref:number> <xref:of> bytes <xref:read.>

readline

Read and return a line from the stream.

If length is specified, at most size bytes will be read.

readline(length=-1)

Parameters

Name	Description
length	Default value: -1

readlines

Return all lines in a file as a list

readlines()

seek

Set current file location

seek(loc, whence=0)

Parameters

Name	Description
loc Required	int byte location
whence	from start of file, current location or end of file, resp. Default value: 0

seekable

Return whether the AzureDLFile is seekable (only in read mode)

seekable()

tell

Current file location

tell()

writable

Return whether the AzureDLFile was opened for writing

writable()

write

Write data to buffer.

Buffer only sent to ADL on flush() or if buffer is bigger than blocksize.

write(data)

Parameters

Name	Description
data Required	bytes Set of bytes to be written.