Awesome tool
Hey this is pretty good. but can we use image to guide pose? like from image to open pose and then apply it to the human figure?
This is the goal; you can send the generated image to the OpenPose preprocessor and generate your image in the reference pose. The component already has the event that triggers the capture.
i actually have little different idea, I want the figure in the 3d space to have that pose. :)
I didn't quite understand what you want. What do you mean by "figure in 3D space"? If you want to change your existing image, I think you'll need to use it as a reference image along with the desired pose image generated in OpenPose to create the variation. This app only aims to demonstrate the functionality of the component for generating a reference pose image; you should integrate this into your workflow.
I updated the application with an OpenPose preprocessor so you can get an idea of what the pose sent to ControlNet will look like.
I think he meant a function to detect pose from provided image in order to manually adjust it in the editor and be able to download it after the modifications.
That would be great. By the way, it would be perfect if the editor could detect multiple poses from an image and allow us to edit more than one pose at a time, so we could simulate interactions between characters (poses). As far as I know, there is not a single open-pose editor available that offers this feature so if you like challenges go for it.. :)
That would be great. By the way, it would be perfect if the editor could detect multiple poses from an image and allow us to edit more than one pose at a time, so we could simulate interactions between characters (poses). As far as I know, there is not a single open-pose editor available that offers this feature so if you like challenges go for it.. :)
I understand, however, what you're suggesting is more complex. Taking an image and mapping it to 3D space would require integrating more than one model into the scene, and I don't know if it would be so easy to manipulate them together. Furthermore, it would require a preprocessor that captures more than one pose in the same photo, like this one: https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch. This application aims only to demonstrate a Gradio component for using standard OpenPose in image generation interfaces. But, perhaps in the future I can invest some time in this new component idea.