Customize Screen Size and Position in Live Stream

[TaiBK15]

Hi @pedroSG94 ,

I would like to customize the screen size and position in the live stream. For example, if my output video size is 1080x1920, I want to adjust the captured screen to a specific size and position within that frame.

How can I achieve this using RootEncoder? Is there an option to set the screen dimensions and position dynamically?

I’ve attached an image for reference.

Thank you for your support!

[pedroSG94]

Hello,

It is not possible by default but you can create a filter to solve it.
Currently you have the BaseObjectFilterRender (used for ImageObjectFilterRender) which is close to your requirement:
https://github.com/pedroSG94/RootEncoder/blob/master/encoder/src/main/java/com/pedro/encoder/input/gl/render/filters/object/BaseObjectFilterRender.java
The idea is only use a texture and modify the scale and position vertex to your requirements.

Remember that filters affect to preview and output image

[TaiBK15]

Thanks for your answer. I will try your solution and check back to see if the requirement is solved.

[TaiBK15]

Could you please share a sample code with me? I tried the code below, but it doesn’t seem to work.

class CustomScreenRender : BaseObjectFilterRender() {
    private var customScaleX = 1f
    private var customScaleY = 1f
    private var offsetX = 0f
    private var offsetY = 0f

    init {
        updateVertices()
    }

    fun setTransform(scaleX: Float, scaleY: Float, offsetX: Float, offsetY: Float) {
        this.customScaleX = scaleX
        this.customScaleY = scaleY
        this.offsetX = offsetX
        this.offsetY = offsetY
        updateVertices()
    }

    private fun updateVertices() {
        val vertices = floatArrayOf(
            -customScaleX + offsetX, -customScaleY + offsetY,
            customScaleX + offsetX, -customScaleY + offsetY,
            -customScaleX + offsetX,  customScaleY + offsetY,
            customScaleX + offsetX,  customScaleY + offsetY
        )
        setPosition(100f, 200f)
    }
}

fun prepareStream(resultCode: Int, data: Intent): Boolean {
      if (!prepare()) {
          return false
      }

      INSTANCE = this
      keepAliveTrick()
      stopService()
      mediaProjection?.stop()
      val mediaProjection = mediaProjectionManager.getMediaProjection(resultCode, data)
      this.mediaProjection = mediaProjection
      val screenSource = ScreenSource(applicationContext, mediaProjection)
      return try {
          genericStream.getGlInterface().setCameraOrientation(0)

          val custom = CustomScreenRender()
          custom.setTransform(0.5f, 0.5f, 0.5f, 0.5f)
          genericStream.getGlInterface().setFilter(custom)
          genericStream.changeVideoSource(screenSource)
          handleMicrophone()
          true
      } catch (ignored: IllegalArgumentException) {
          false
      }
  }

[TaiBK15]

I've been experimenting with adjusting the output video size and found the following code works for resizing, which is great!
genericStream.getGlInterface().setEncoderSize(width.toInt(), height.toInt())

However, I'm also looking to adjust the position of the recorded video frame within the output. I noticed that the drawEncoder method in the GlStreamInterface class (code snippet below) only takes width and height parameters, but no parameters for the X and Y position. It appears that the GLES20.glViewport() function determines the area where the drawing occurs. Currently, the viewport.getX() and viewport.getY() seem to be calculated based on the width, height, likely resulting in a fixed position.

public void drawEncoder(int width, int height, boolean isPortrait, int rotation,
      boolean flipStreamVertical, boolean flipStreamHorizontal) {
    GlUtil.checkGlError("drawScreen start");

    updateMatrix(rotation, SizeCalculator.calculateFlip(flipStreamHorizontal, flipStreamVertical), MVPMatrix);
    ViewPort viewport = SizeCalculator.calculateViewPortEncoder(width, height, isPortrait);
    GLES20.glViewport(viewport.getX(), viewport.getY(), viewport.getWidth(), viewport.getHeight());

    draw();
  }

To achieve the ability to change the video's position in the output, would it be possible to introduce additional logic or modify the drawEncoder method (or perhaps introduce a new method) to accept parameters for the X and Y coordinates? This would allow for more flexible control over the final video layout.

Thanks for considering this!

[pedroSG94]

Hello,

It is possible. I will allow send a ViewPort class as parameter and calculate if myself if the parameter is null.
In this case you will have to calculate all values yourself (x, y, width, height)

[TaiBK15]

Hi @pedroSG94,
Hope you're well.
Following up on the approach you suggested for adding the ViewPort as parameter. I just want to check if there's any update on that solution.
Thanks again for your support!

[pedroSG94]

Hello,

It is now available in the last commit of master branch

[TaiBK15]

Hi @pedroSG94 ,

Thanks for adding the viewport parameter functionality in the latest master commit!

I've integrated the changes, and the main video stream correctly adjusts to the specified viewport – that part works great.

However, I noticed that applying the viewport also affects the overlay layer along with the video. My goal is to keep the overlay rendering at full screen size, independent of the video's viewport adjustments.

Do you have any suggestions on how I might achieve this? Is there a way to apply the viewport transformation only to the video stream itself without affecting the overlay?

Thanks again for your help!

[pedroSG94]

Hello,

It is not possible using a viewport because the viewport affect to all.
If you want do it you will have to do a custom filter, add it before the overlay filter (position 0 custom filter and position 1 overlay filter) and modify matrix to do it.

[TaiBK15]

Hello @pedroSG94,
Thank you for clarifying that the viewport affects everything and for suggesting the custom filter approach. I understand the logic of placing filter before the overlay and modifying the matrix.

But honestly, I'm still a bit unsure about the specifics of implementing this within the RootEncoder framework, especially regarding the matrix manipulation part inside the filter. I tried setting one up, but haven't cracked it yet.

Would you be able to provide a small code example demonstrating how to create such a custom filter and modify the matrix to achieve a viewport-like effect (scale/translate) on the video without affecting the overlay? Seeing how you'd structure it would be a great help for me to follow your instructions correctly.

Thanks again for your time and support!

[pedroSG94]

Hello,

You can copy this class:
https://github.com/pedroSG94/RootEncoder/blob/master/encoder/src/main/java/com/pedro/encoder/input/gl/render/filters/CropFilterRender.kt

This is a filter with is used to crop and you can modify setCropArea method as you want instead of crop a rectangle.
Now, you only need add both filters (custom and overlay) to the stream in the correct position:

    genericStream.getGlInterface().addFilter(filterPosition = 0, customFilter)
    genericStream.getGlInterface().addFilter(filterPosition = 1, overlayFilter)

This is working like layers in photoshop. Higher position affect to lower positions but lower position no affect to higher positions

[TaiBK15]

Hi @pedroSG94,
I have tried your instruction and it has worked great.
I am really appreciate for your support.
Thanks again for your time.

[pedroSG94]

Closing as resolved

[TaiBK15]

Hi @pedroSG94 ,

I want to resize only the screen recording content inside the video output, not the black bars on the sides. My device resolution is 1080x2168, but the output is 1080x1920 (9:16), so black bars appear. When I scale, everything including black bars gets resized — I only want to resize the screen content.

It would be great if we can set custom size and position for screen capture, separate from output resolution.

Can you suggest a way to do that?

Thanks for your support!