jaulib v1.4.0-2-g788cf73
Jau Support Library (C++, Java, ..)
|
Input and Output (IO) types and functionality. More...
Namespaces | |
namespace | jau::io::http |
namespace | jau::io::impl |
Classes | |
struct | jau::io::AsyncStreamResponse |
Asynchronous stream response. More... | |
class | jau::io::Bitstream |
Versatile Bitstream implementation supporting: More... | |
class | jau::io::ByteInStream_Feed |
Ringbuffer-Based byte input stream with an externally provisioned data feed. More... | |
class | jau::io::ByteInStream_URL |
Ringbuffer-Based byte input stream with a URL connection provisioned data feed. More... | |
class | jau::io::ByteStream |
Byte stream interface. More... | |
class | jau::io::ByteStream_File |
File based byte input stream, including named file descriptor. More... | |
class | jau::io::ByteStream_Recorder |
Wrapped byte input stream with the capability to record the read byte stream at will. More... | |
class | jau::io::ByteStream_SecMemory |
Secure Memory-Based byte input stream. More... | |
class | jau::io::IOStateCap |
Supporting std::basic_ios's iostate capabilities for all ByteStream implementations. More... | |
struct | jau::io::SyncStreamResponse |
Synchronous stream response. More... | |
class | jau::io::url_header_resp |
Synchronized URL header response completion as used by asynchronous read_url_stream(). More... | |
Typedefs | |
typedef jau::function< bool(AsyncStreamResponse &, const uint8_t *, size_t, bool)> | jau::io::AsyncStreamConsumerFunc |
Asynchronous stream consumer function. | |
using | jau::io::AsyncStreamResponseRef = std::shared_ptr<AsyncStreamResponse> |
typedef jau::ringbuffer< uint8_t, size_t > | jau::io::ByteRingbuffer |
using | jau::io::net_tk_handle = void* |
typedef jau::ordered_atomic< io_result_t, std::memory_order_relaxed > | jau::io::relaxed_atomic_io_result_t |
typedef std::basic_string< char, std::char_traits< char >, jau::callocator_sec< char > > | jau::io::secure_string |
template<typename T> | |
using | jau::io::secure_vector = std::vector<T, jau::callocator_sec<T>> |
typedef jau::function< bool(secure_vector< uint8_t > &, bool)> | jau::io::StreamConsumerFunc |
Stream consumer function. | |
typedef jau::function< bool(SyncStreamResponse &, const uint8_t *, size_t, bool)> | jau::io::SyncStreamConsumerFunc |
Synchronous stream consumer function. | |
using | jau::io::SyncStreamResponseRef = std::shared_ptr<SyncStreamResponse> |
Functions | |
net_tk_handle | jau::io::create_net_tk_handle () noexcept |
creates a reusable handle, free with free_net_tk_handle() after use. | |
void | jau::io::free_net_tk_handle (net_tk_handle handle) noexcept |
frees a handle after use created by create_net_tk_handle() | |
std::string_view | jau::io::uri_tk::get_scheme (const std::string_view &uri) noexcept |
Returns the valid uri-scheme from given uri, which is empty if no valid scheme is included. | |
bool | jau::io::uri_tk::is_httpx_protocol (const std::string_view &uri) noexcept |
Returns true if the uri-scheme of given uri matches the http or https protocol, i.e. | |
bool | jau::io::uri_tk::is_local_file_protocol (const std::string_view &uri) noexcept |
Returns true if the uri-scheme of given uri matches the local file protocol, i.e. | |
jau::io::JAU_MAKE_BITFIELD_ENUM_STRING (iomode_t, read, write, atend, trunc) | |
jau::io::JAU_MAKE_BITFIELD_ENUM_STRING (iostate_t, goodbit, badbit, eofbit, failbit, timeout) | |
std::ostream & | jau::io::operator<< (std::ostream &os, const Bitstream &v) |
std::ostream & | jau::io::operator<< (std::ostream &os, const ByteStream &v) |
std::ostream & | jau::io::operator<< (std::ostream &os, io_result_t v) |
void | jau::io::print_stats (const std::string &prefix, const uint64_t &out_bytes_total, const jau::fraction_i64 &td) noexcept |
bool | jau::io::uri_tk::protocol_supported (const std::string_view &uri) noexcept |
Returns true if the uri-scheme of given uri matches a supported by libcurl network protocols otherwise false. | |
uint64_t | jau::io::read_file (const std::string &input_file, secure_vector< uint8_t > &buffer, const StreamConsumerFunc &consumer_fn) noexcept |
Synchronous byte input stream reader from given file path using the given StreamConsumerFunc consumer_fn. | |
uint64_t | jau::io::read_stream (ByteStream &in, secure_vector< uint8_t > &buffer, const StreamConsumerFunc &consumer_fn) noexcept |
Synchronous byte input stream reader using the given StreamConsumerFunc consumer_fn. | |
uint64_t | jau::io::read_stream (ByteStream &in, secure_vector< uint8_t > &buffer1, secure_vector< uint8_t > &buffer2, const StreamConsumerFunc &consumer_fn) noexcept |
Synchronous double-buffered byte input stream reader using the given StreamConsumerFunc consumer_fn. | |
uint64_t | jau::io::read_url_stream (const std::string &url, secure_vector< uint8_t > &buffer, const StreamConsumerFunc &consumer_fn) noexcept |
Synchronous URL stream reader using the given StreamConsumerFunc consumer_fn. | |
AsyncStreamResponseRef | jau::io::read_url_stream_async (net_tk_handle handle, const std::string &url, http::PostRequestPtr httpPostReq, ByteRingbuffer *buffer, const AsyncStreamConsumerFunc &consumer_fn) noexcept |
Asynchronous URL stream reader using the given AsyncStreamConsumerFunc consumer_fn. | |
SyncStreamResponseRef | jau::io::read_url_stream_sync (net_tk_handle handle, const std::string &url, http::PostRequestPtr httpPostReq, ByteRingbuffer *buffer, const SyncStreamConsumerFunc &consumer_fn) noexcept |
Synchronous URL stream reader using the given SyncStreamConsumerFunc consumer_fn. | |
std::vector< std::string_view > | jau::io::uri_tk::supported_protocols () noexcept |
Returns a list of supported protocol supported by libcurl network protocols, queried at runtime. | |
std::unique_ptr< ByteStream > | jau::io::to_ByteInStream (const std::string &path_or_uri, jau::fraction_i64 timeout=20_s) noexcept |
Parses the given path_or_uri, if it matches a supported protocol, see jau::io::uri::protocol_supported(), but is not a local file, see jau::io::uri::is_local_file_protocol(), ByteInStream_URL is being attempted. | |
std::string | jau::io::to_string (const ioaccess_t v) noexcept |
Return std::string representation of the given ioaccess_t. | |
std::string | jau::io::toString (io_result_t v) noexcept |
Variables | |
const size_t | jau::io::BEST_URLSTREAM_RINGBUFFER_SIZE = 2_uz * 16384_uz |
Input and Output (IO) types and functionality.
using jau::io::secure_vector = std::vector<T, jau::callocator_sec<T>> |
Definition at line 33 of file io_util.hpp.
typedef std::basic_string<char, std::char_traits<char>, jau::callocator_sec<char> > jau::io::secure_string |
Definition at line 35 of file io_util.hpp.
typedef jau::ringbuffer<uint8_t, size_t> jau::io::ByteRingbuffer |
Definition at line 37 of file io_util.hpp.
typedef jau::ordered_atomic<io_result_t, std::memory_order_relaxed> jau::io::relaxed_atomic_io_result_t |
Definition at line 65 of file io_util.hpp.
typedef jau::function<bool(secure_vector<uint8_t>& , bool )> jau::io::StreamConsumerFunc |
Stream consumer function.
bool consumer(secure_vector<uint8_t>& data, bool is_final)
Returns true to signal continuation, false to end streaming.
Definition at line 82 of file io_util.hpp.
using jau::io::net_tk_handle = void* |
Definition at line 223 of file io_util.hpp.
using jau::io::SyncStreamResponseRef = std::shared_ptr<SyncStreamResponse> |
Definition at line 267 of file io_util.hpp.
typedef jau::function<bool(SyncStreamResponse& , const uint8_t* , size_t , bool )> jau::io::SyncStreamConsumerFunc |
Synchronous stream consumer function.
bool consumer(AsyncStreamResponse& resp, const uint8_t* data , size_t len, bool is_final)
Returns true to signal continuation, false to end streaming.
Definition at line 275 of file io_util.hpp.
using jau::io::AsyncStreamResponseRef = std::shared_ptr<AsyncStreamResponse> |
Definition at line 348 of file io_util.hpp.
typedef jau::function<bool(AsyncStreamResponse& , const uint8_t* , size_t , bool )> jau::io::AsyncStreamConsumerFunc |
Asynchronous stream consumer function.
bool consumer(AsyncStreamResponse& resp, const uint8_t* data , size_t len, bool is_final)
Returns true to signal continuation, false to end streaming.
Definition at line 356 of file io_util.hpp.
|
strong |
I/O read or write access.
Enumerator | |
---|---|
read | Read intent. |
write | Write intend. |
Definition at line 41 of file bit_stream.hpp.
|
strong |
Definition at line 41 of file byte_stream.hpp.
|
strong |
Stream I/O mode, e.g.
read and/or write.
Definition at line 63 of file byte_stream.hpp.
|
strong |
I/O direction, read or write.
Enumerator | |
---|---|
READ | Read Operation. |
WRITE | Write Operation. |
Definition at line 44 of file io_util.hpp.
|
strong |
I/O operation result value.
Enumerator | |
---|---|
FAILED | Operation failed. |
NONE | Operation still in progress. |
SUCCESS | Operation succeeded. |
Definition at line 55 of file io_util.hpp.
|
noexcept |
Return std::string representation of the given ioaccess_t.
v | the ioaccess_t value |
Definition at line 920 of file byte_stream.cpp.
|
inline |
Definition at line 789 of file bit_stream.hpp.
jau::io::JAU_MAKE_BITFIELD_ENUM_STRING | ( | iostate_t | , |
goodbit | , | ||
badbit | , | ||
eofbit | , | ||
failbit | , | ||
timeout | ) |
|
inline |
Definition at line 639 of file byte_stream.hpp.
|
noexcept |
Parses the given path_or_uri, if it matches a supported protocol, see jau::io::uri::protocol_supported(), but is not a local file, see jau::io::uri::is_local_file_protocol(), ByteInStream_URL is being attempted.
If the above fails, ByteStream_File (read-only) is attempted.
If non of the above leads to a ByteStream without ByteStream::fail(), nullptr is returned.
path_or_uri | given path or uri for with a ByteInStream instance shall be established. |
timeout | in case path_or_uri resolves to ByteInStream_URL, timeout is being used as maximum duration to wait for next bytes at ByteInStream_URL::available(), defaults to 20_s |
Definition at line 693 of file byte_stream.cpp.
|
inlinenoexcept |
|
inline |
Definition at line 74 of file io_util.hpp.
|
noexcept |
Synchronous byte input stream reader from given file path using the given StreamConsumerFunc consumer_fn.
To abort streaming, user may return false
from the given consumer_func
.
It is guaranteed that consumer_fn() is called with is_final=true
once at the end, even if input_file
stream has zero size.
input_file | input file name path, - denotes std::stdin. |
buffer | secure std::vector buffer, passed down to consumer_fn |
consumer_fn | StreamConsumerFunc consumer for each received heap of bytes, returning true to continue stream of false to abort. |
Definition at line 47 of file io_util.cpp.
|
noexcept |
Synchronous byte input stream reader using the given StreamConsumerFunc consumer_fn.
To abort streaming, user may return false
from the given consumer_func
.
It is guaranteed that consumer_fn() is called with is_final=true
once at the end, even input stream has zero size.
in | the input byte stream to read from |
buffer | secure std::vector buffer, passed down to consumer_fn |
consumer_fn | StreamConsumerFunc consumer for each received heap of bytes, returning true to continue stream of false to abort. |
Definition at line 60 of file io_util.cpp.
|
noexcept |
Synchronous double-buffered byte input stream reader using the given StreamConsumerFunc consumer_fn.
To abort streaming, user may return false
from the given consumer_func
.
It is guaranteed that consumer_fn() is called with is_final=true
once at the end, even if input stream has zero size.
Implementation reads one buffer ahead in respect to consumer_fn().
If reading zero bytes on the next buffer, it propagates the end-of-file (EOF) to the previous buffer which will be send via consumer_fn() next.
This way, the consumer_fn() will always receive its is_final
flag on the last sent bytes (size > 0) even if the content-size is unknown (pipe).
Hence it allows e.g. decryption to work where the final data chunck must be processed as such.
in | the input byte stream to read from |
buffer1 | secure std::vector buffer, passed down to consumer_fn |
buffer2 | secure std::vector buffer, passed down to consumer_fn |
consumer_fn | StreamConsumerFunc consumer for each received heap of bytes, returning true to continue stream of false to abort. |
Definition at line 101 of file io_util.cpp.
|
noexcept |
Synchronous URL stream reader using the given StreamConsumerFunc consumer_fn.
To abort streaming, user may return false
from the given consumer_func
.
Standard implementation uses curl, hence all libcurl network protocols are supported, see jau::io::uri::supported_protocols().
If the uri-sheme doesn't match a supported protocol, see jau::io::uri::protocol_supported(), function returns immediately with zero bytes.
Environment variables:
jau_io_net_ssl_verifypeer=[true|false]
to enable or disable SSL peer verification. Defaults to true
.jau_io_net_verbose=[true|false]
to enable or disable verbose on stderr stream communication. Defaults to false
.url | the URL to open a connection to and stream bytes from |
buffer | secure std::vector buffer, passed down to consumer_fn |
consumer_fn | StreamConsumerFunc consumer for each received heap of bytes, returning true to continue stream of false to abort. |
Definition at line 312 of file io_util.cpp.
|
noexcept |
creates a reusable handle, free with free_net_tk_handle() after use.
Definition at line 1072 of file io_util.cpp.
|
noexcept |
frees a handle after use created by create_net_tk_handle()
Definition at line 1081 of file io_util.cpp.
|
noexcept |
Synchronous URL stream reader using the given SyncStreamConsumerFunc consumer_fn.
Function returns after completion.
To abort streaming, (1) user may return false
from the given consumer_func
. Asynchronous URL read content using the given byte jau::ringbuffer, allowing parallel reading.
To abort streaming, (2) user may set given reference results
to a value other than async_io_result_t::NONE.
Standard implementation uses curl, hence all libcurl network protocols are supported, see jau::io::uri::supported_protocols().
If the uri-sheme doesn't match a supported protocol, see jau::io::uri::protocol_supported(), SyncStreamResponse::failed() returns true.
Environment variables:
jau_io_net_ssl_verifypeer=[true|false]
to enable or disable SSL peer verification. Defaults to true
.jau_io_net_verbose=[true|false]
to enable or disable verbose on stderr stream communication. Defaults to false
.handle | optional reused user-pooled net toolkit handle, see create_net_tk_handle(). Pass nullptr to use an own local handle. |
url | the URL to open a connection to and stream bytes from |
httpPostReq | optional HTTP POST request data, maybe nullptr |
buffer | optional jau::ringbuffer<uint8_t>, if not nullptr will be synchronously filled with received data |
consumer_fn | SyncStreamConsumerFunc consumer for each received heap of bytes, returning true to continue stream of false to abort. |
Definition at line 1093 of file io_util.cpp.
|
noexcept |
Asynchronous URL stream reader using the given AsyncStreamConsumerFunc consumer_fn.
Function returns immediately.
To abort streaming, (1) user may return false
from the given consumer_func
. Asynchronous URL read content using the given byte jau::ringbuffer, allowing parallel reading.
To abort streaming, (2) user may set given reference results
to a value other than async_io_result_t::NONE.
Standard implementation uses curl, hence all libcurl network protocols are supported, see jau::io::uri::supported_protocols().
If the uri-sheme doesn't match a supported protocol, see jau::io::uri::protocol_supported(), AsyncStreamResponse::failed() returns true.
Environment variables:
jau_io_net_ssl_verifypeer=[true|false]
to enable or disable SSL peer verification. Defaults to true
.jau_io_net_verbose=[true|false]
to enable or disable verbose on stderr stream communication. Defaults to false
.handle | optional reused user-pooled net toolkit handle, see create_net_tk_handle(). Pass nullptr to use an own local handle. |
url | the URL to open a connection to and stream bytes from |
httpPostReq | optional HTTP POST request data, maybe nullptr |
buffer | optional jau::ringbuffer<uint8_t>, if not nullptr will be asynchronously filled with received data |
consumer_fn | AsyncStreamConsumerFunc consumer for each received heap of bytes, returning true to continue stream of false to abort. |
Definition at line 1119 of file io_util.cpp.
|
noexcept |
|
noexcept |
Returns a list of supported protocol supported by libcurl network protocols, queried at runtime.
Definition at line 160 of file io_util.cpp.
|
noexcept |
Returns the valid uri-scheme from given uri, which is empty if no valid scheme is included.
The given uri must include at least a colon after the uri-scheme part.
uri | an uri |
Definition at line 183 of file io_util.cpp.
|
noexcept |
Returns true if the uri-scheme of given uri matches a supported by libcurl network protocols otherwise false.
The uri-scheme is retrieved via get_scheme() passing given uri, hence must include at least a colon after the uri-scheme part.
The libcurl supported protocols is queried at runtime, see supported_protocols().
uri | an uri to test |
Definition at line 195 of file io_util.cpp.
|
noexcept |
Returns true if the uri-scheme of given uri matches the local file
protocol, i.e.
starts with file://
.
uri | an uri to test |
Definition at line 205 of file io_util.cpp.
|
noexcept |
Returns true if the uri-scheme of given uri matches the http
or https
protocol, i.e.
starts with http:
or https:
.
uri | an uri to test |
Definition at line 209 of file io_util.cpp.
|
extern |
Definition at line 78 of file byte_stream.cpp.