Repository navigation
Why implement Buffer and Memory?聽#125
Description
Activity
Actually,
IOBufferdoesn't work so well because it uses the same pointer when reading/writing. What we want is a buffered stream.Maybe it makes sense to use
BufferedStreams.jlsoTranscodingStreams.jldoesn't need to ship its own fast buffer implementation? One problem is thatBufferedStreams.jlrequires choosing betweenBufferedInputStreamandBufferedOutputStreamstructs. This doesn't seem compatible with the currentTranscodingStreamdesign which uses the given stream as either input or output depending on whether it's in:reador:writemode.I think it's better design to have two separate structs for input and output transcoding, instead of keeping this logic inside some state component. It would also make the logic simpler since we don't have to worry about writing while in read state or vice versa.
Any thoughts? Maybe @jakobnissen ?
It seems complex to me too, but my experience is that usually stuff is complicated because there are edge cases I havne't thought of.
Memoryis a way to abstract over input memory, which may not be in the form of aVector{UInt8}, i.e. it may be aSubString{String}. It may be possible to instead make it a parameterized Julia type instead of storing the raw pointer.I don't think we can remove
Buffer. We need low-level control of the implementation, and so would have to rely on implementation details ofIOBuffer. We also need more features likeBuffer.marked, whichIOBufferdoes not provide.
Hey I've just been poking around this package and it seems overly complex. Can someone more familiar with it make sure I haven't just missed Chesterton's fence 馃槢 Happy to work on this if people agree it makes sense
AFAICT,
TranscodingStreammanages an input and outputBuffer; data is read from the input stream into the inputBuffer. This is passed to a codec viaMemory(which is just an abstraction over the "used" and "free" parts of aBuffer). The codec writes to the outputBuffer, and that's given to the user via an output stream when requested.Memory. I can't figure out why this is useful, aside from just making things slightly easier for downstream codecs that are C-based and need pointers. If that's the case, we could views instead.Buffer. We really don't need our own implementation; I thinkIOBuffershould work. Fundamentally, we just want to load data from the input stream into some input array, let a codec convert part of that to an output array, and send it to the user via a stream.Less importantly:
Error. Why not justtry/catchinstead of passing yet another variable around as state?