artificial intelligence in working with the render.

[GaudiGalopin3324]

I must say right away that I am not an expert in this field, rather I want to get advice from knowledgeable users. My attempts to study this issue are still weak. Dear Keet, he correctly noted that the method of replacing 3D models with images on boxes has an important drawback: you can't "walk around" the room, the box only works from one point of view. But I tried using neural networks. Here's what I got and here are my thoughts. The main difficulty in working with artificial intelligence is that it is very difficult to get the desired result. There are probably specialists who can do everything, but I mostly got surprises. AI does everything with unexpected options, even if the promt is written in great detail and clearly (as it seems to a living person). For now, I've settled on the Chinese startup HAILUO AI. It has several very important advantages over its competitors. First of all, incredible generosity. A beginner is given a very large margin for training renders, it was enough for me to work very hard with video generation for two days. There is no such thing anywhere, everywhere developers are very greedy. During this free limit, I was able to figure out the possibilities and make almost 30 interesting videos for myself. That's a lot!! Now I can tell you my conclusions.
The second important advantage is that there is a generation option with the ability to control camera movement. This is very important because it allows you to do something consciously, at least partially. Rather than hoping for a miracle from a neural network without guarantees. This generation method is called 12V-01-Director. Camera movements can be of several types. To work with interior photography, you can take one view or combine several views. For example, moving to the side, turning your head, lifting the camera up, and other combinations. You can make more complex combinations, but the clip time is always limited to 6 seconds. My first experience stunned me. Artificial intelligence was able to recognize all the individual objects in the photo, distribute them in virtual space and even looked behind your back, sometimes very correctly!! Unbelievable. Here's an example of this render.



A 6-second clip with a camera consisting of four types of movement (left shift, right turn, upward tilt) looks like this. Pay attention to the chair!! And the mirror on the ceiling!!! There are reflections that are almost right!! And it's originally just a photo, it's not a group of 3D models. Clip 1 is simple
to set up the camera, it is important not to go much beyond the original frame during these 6 seconds, otherwise the intelligence will come up with something unexpected, most likely unnecessary and incorrect. Therefore, I advise you to combine the opposite movements - a shift to the left, then a turn of the head to the right, so that the center of the photo remains in place.

I'll continue in the next post.

[GaudiGalopin3324]

you can move in different directions, but be prepared that if the camera goes beyond the boundaries of the photo, objects that do not exist in the project will appear. I wrote a promt and put a girl with a book in a chair. She found herself in a completely different place and in a different chair. This is normal for AI [:D] Clip 2

Adding people and animals is certainly very exciting, but it's completely unpredictable. People can pop out of the closet and disappear into the couch. Or suddenly jump out the window. I didn't want that!!! So for now, it's safest to be without characters. Even if you write everything in great detail, there will still be a surprise, and on the second attempt with the same promt everything will be different, there are no guarantees. But the pieces of furniture are more or less similar to the truth, it's amazing! Such a video render in SH3D with lighting and in good quality can take a week to upload. And this video is 15-2 minutes long.

Clip 3

Clip 4

Clip 5

Clip 6

Clip 7

But sometimes things get completely out of control. [:D] [:D] [:D]

Clip 8

Clip 9 [:))] [:D] [:D]

Of course, I am interested in your opinion and your experience. There are probably more efficient platforms with AI [:x]

[Keet]

Very interesting results! So far I haven't done anything with any AI, I'm too old to add yet another technology to my working set. I'm trying to use less, not more [:))]

If I understand it correct from what you wrote: you take a photo and give movement instructions to create the video. Correct?

If I'm not mistaking the Sweet Home 3D virtual tour video is created by generating transitions between two or more photo's, i.e. saved camera positions. I think the plan is used to calculate the transitions and uses a lot of zoom to simulate movement.
Maybe AI can generate transitions between two photo's, a little like the old-time morphing software did 20-30 years ago but now with movement from one to the other. It will probably need a different AI but it could be a (better?) alternative for the long creation times of a Sweet Home 3D video. Then stitch the different videos together to get a longer video.

Your initial experiments are very promising, especially since you only started it a few days ago. I'm looking forward to seeing more!

[GaudiGalopin3324]

Hi Keet. Here I offer AI one of my photos. And I suggest the camera path. Then the AI processes everything and outputs this video. It separates objects in a photo, determines where they are in space, and can even restore the reverse side of objects!! It's unbelievable! In one such video, he made the back of the sofa with neat stitching, which is fantastic. I even assumed that he had identified the real model of the sofa and made it right))). Of course I don't know. But the result is very interesting and most importantly it is not difficult to do it. But it is with this platform, where there is a clear interface. The possibilities of this method are very modest in time, only 6 seconds and it takes a lot of attempts for the correct result. Of course, AI is developing now, and probably in a year or two there will be nothing but AI. Sadly. But there's still time to laugh. And you need not to lose it, to spend these fun days usefully. [:D]

[Keet]

I wonder if there's an AI that can create an OBJ/MTL 3D model from a photo. Even if not perfect it can be a base that we can edit.
When you say that the AI created the back of the couch with stitching's it shows that the AI can create an object in 3D. Professional 3D model developers will not be happy if this becomes so easy!

[GaudiGalopin3324]

We have a whole year ahead of us!! [:D] [:D] [:D]

[dorin]

@Keet
More than 10 years ago I've used an online tool 3D Face Reconstruction.
Based on a photo produce a 3D object of my face.
Try to find something similar on the net. What I've used in the past it appear to not be available no more.
The results was pretty good for my ugly face.

[GaudiGalopin3324]

Unbelievable! I found an interesting Trellis website. Based on the photo of the chair, I got an excellent 3D model (in 30 seconds), but in some obscure GLB format. I'm sure you need to search, and there will be for OBJ. or try to convert. Yes, I converted, it turned out to be simple. Unbelievable!!! You can get the model from the photo now!!!

Trellis

the original photo



and this is already a model in SH3D!!!!



I probably won't sleep tonight. It took less than 5 minutes for everything!! You just need a photo!! [:D] [:D] [:D] [:D] [:x]

[Keet]

I probably won't sleep tonight. It took less than 5 minutes for everything!! You just need a photo!! [:D] [:D] [:D] [:D] [:x]

That would be a realy good solution if photo-on-a-box doesn't work out or when it is way too much work.
Now try the cabinet you and captaincook spend so much time on to get the images correct for use on the box [:D]

[GaudiGalopin3324]

I am disappointed by people's greed again. A tiny free quota.. Why don't people want to be selfless..? Just be happy with the word "Thank you.". I'll try to make a chest of drawers tomorrow, when they give me the opportunity)). The only difficulty there is that the texture can only be like in the photo, and you can't change it in parts. You can only completely change the color. I only managed to make a Cattelan Penguin coffee table [:)]

[:">]