Implement texture mipmaps to reduce texture aliasing/noise

Project:YafaRay
Version:development
Component:Code
Category:feature request
Priority:critical
Assigned:David Bluecame
Status:ready to commit
Description

hi David,

you'll find that I have attached two examples of renderings. One using the Yafaray engine and the other in a Real Time rendering engine both using same textures, etc.. Now have a look at these two images. In "Bad_mipmaps" image (produced in yafaray) you can notice the added noise to the image in comparison to what we have obtained with the Real Time engine. We have noticed that if we increase the number of minsamples, rendering time becomes exponential and merely unacceptable. Do you have any suggestions on how to solve this problem? Do we have access to various levels of mipmapping?

I'm also attaching a rendered global scene where he above problem is highly evident together with the xml file of Good_ and Bad_ test case.

Best Regards

John

ACCA Software S.p.A.

AttachmentSize
Bad_MipMaps_Yafaray_E_1.1.0.jpg787.59 KB
Good_MipMaps_RealTime_Edificius.jpg1.21 MB
GlobalSceneMipMaps_error.jpg1014.77 KB
MipMapsProblem_XML.zip1.91 MB

Comments

#1

Title:MipMaps PROBLEM!» MipMaps problem

Hello, John.

The parameter bump_strength is too high (4.0). Please try to use bump_strength values in the range 0.5-1.0 and let me know if it helps.

#2

Hi,

thanks for your reply. We have tried reducing bump_strength but noise still appears in renders. While testing our models in Blender, setting the above paramter to 1, we also notice that this value is increased to 2 in the relating .xml file. Is this normal?

As a further detail that we think may help, we have also tried rendering the test file WITHOUT the NORMAL but you'll see that the noise effect still appears.

Could it be the case of a mip mapping error??

 

Sending you extra attachements to show you what we are obtaining (Good_ / Bad_ / .xml file)

Regards

John

ACCA Software S.p.A.

AttachmentSize
Bad_MipMaps_Yafaray_NONormal_E_1.1.0.jpg 863.81 KB
Good_MipMaps_RealTime_NONormal_Edificius.jpg 1.68 MB
MipMapsProblem_NONormal_XML.zip 1.27 MB

#3

Hi, David

Nothing new about the problem mipmaps ???
In your opinion is our problem or not ???

John

#4

Status:active» needs more infos

Hello,

Honestly I don't see anything wrong in YafaRay here.

The texture OD_CACB.jpg is very noisy. In fact, I'm surprised that you have so little noise in the Edificus realtime engine, because it's not showing the noise from the texture. So, maybe Edificus is not using the texture at 100% but probably "diluting" it a bit.

You can also "dilute" a bit the texture in YafaRay, you can do it in two ways:

* For bump/normal mapping tweaking the parameter "bump_strenght" (you have 4 which is too much I think)

* For the texture itself you can dilute it by tweaking parameter "colfac", for example:

   <diffuse_shader sval="material_diffuse_layer_node" />
   <list_element>
      <name sval="material_diffuse_layer_node" />
      <mode ival="2" />
      <def_col r="1" g="1" b="1" a="1" />
      <def_val fval="1" />
      <colfac fval="0.7" />

Please let me know if you can get better results decreasing those two values.

#5

Hello,

Please have you done the tests described above?  I need to know if this can be solved by adjusting the scene parameters or if further work is needed in the code.

#6

Status:needs more infos» needs review

#7

Hello David,

For this type of problem there is news in the new version ???

This problem for us is very important because you get rendering ugly and we have not yet found a solution !!!

Please if you have some new ideas about how to solve let us know, so we do tests to see if we find the solution !!!

regards
John

 
 
 

#8

Please will you send me a sample Edificius scene with this problem?   A simple scene, as long as it shows the problem at 2 or 3 elements should be enough.

#9

 

#10

Hello David,

I could not send the file here because it is larger than 2MB.
I gave you the files in private, let me know if it's okay!

regards
John

#11

Title:MipMaps problem» Implement texture mipmaps to reduce texture aliasing/noise
Category:bug report» feature request
Priority:normal» critical
Assigned to:Anonymous» David Bluecame
Status:needs review» active

Hello, John.

First of all, I have to say that I've been investigating this issue (intermitently, of course) for a long time, but I could never find anything wrong in YafaRay integration, sampling or texture sampling/mapping processes.

However, by using Edificius I could see the same difference you mention, but I could never point out what the problem was.

I believe I found the answer at last. The reason I could not find the problem is because I was looking for a bug. However that's not the case here. I believe the problem is that YafaRay lacks the ability to produce texture mipmaps with varying resolutions depending on the distance to the textured objects.

As YafaRay is sampling the same texture no matter if the object is far or near, when the object is far away sampling the original texture can produce high frequency sampling artifacts, including noise, which is the difference you are experiencing in Edificius, and I'm sure we are also suffering it in other rendering processes including Blender, Wings3D and others that use YafaRay.

So, I'm turning this bug report into a Feature Request to implement mipmap support in YafaRay. It will not be easy but now that I understand the problem at last I hope to be able to do something about it.

Curiously, for a full mipmap implementation there is part of the code that seems already implemented (the Ray differentials calculations), although I could never see it was used in any rendering operations nor texture sampling. I'm not sure if differentials are correctly implemented but it's a place to start.

I have two possible choices now:

* I could try implementing a fully mipmap support dependant on Ray differentials, which would be ideal, but will take some time.

* Or I could implement in the short term a "dirty trick", which is adding a simple functionality that allows to use the full texture for short distances and a simple average color of the texture for long distances and a "blend" of both for intermediate distances (being the min/max distances configurable). This could perhaps help in the short term while I try implementing the full mipmap support in the medium term.

In any case, thank you for bringing this problem to our attention. Once mipmaps are implemented I'm sure it will help improving the quality of the renders and reduce the render times.

Please let me know if the short term "dirty trick" is good for you in the short term or if you prefer to wait for the full mipmap implementation.

Best regards!

#12

I have a prototype ready with a distance averaging. It's far from ideal as it would require user setting of the parameters in many cases and will only cover trivial cases with textures like asphalt but not more general texture cases, reflections, etc, which will need a full mipmap/differential rays implementation (that hopefully I will try to do in the medium term)

In any case, I want to show you an example of this potential new feature.

YafaRay - 1 - texture processing without distance averaging.pngYafaRay - 1 - texture processing without distance averaging.png

 

YafaRay - 2 - texture processing with distance averaging.pngYafaRay - 2 - texture processing with distance averaging.png

#13

hi David,

we're extremely happy to have contributed in highlighting this kind of problem and therefore help improve Yafaray.

With regard to your "dirty trick" ( :-))  allow me to give it a try and I'll let you know how it goes.

If you don't mind, although I'm not an expert in this particular field, I'd like to suggest an alternative. It may sound stupid, but here it is:

How about doing a pre-calc before launching the rendering process as follows:

- calculate the distance of each object from the camera by simply calculating the the camera - object bounding-box vector.

In relation to these calculated distances, assign the appropriate level of mip-mapping to be used. Therefore becoming a scalable function and during the raycasting process you already know which level of mip mapping should be assigned.

This obviously means that the mip maps have to be pre-calculated in Yafaray but this would result to a simple bilinear scaling operation of the texture at nominal resolution.

Although not an optimal solution, it may serve the purpose for most cases.

I'll let you know how your trick works out and will definitely be useful for us at this stage.

 

Best Regards

John - ACCA Software S.p.A.

#14

Priority:critical» normal
Status:active» needs work

Thank you, John. Yes, indeed automatic mipmap generation would be ideal, but that will take time to be implemented.

For now as a "quick fix" this is what I will implement. I hope it helps for the time being until full mipmap generation gets implemented.

New texture functionality: distance color averaging
-----------------------------------------------------------------------------

As described in http://www.yafaray.org/node/695 there is a limitation in the current YafaRay code respect to textures.

YafaRay samples the textures "as is", independently of the distance between the camera and the object. That causes far objects to show noise in the texture due to high-frequency sampling problems (texture aliasing).

Normally this gets mitigated by increasing the number of samples. However, that's a very inefficient way and takes a very long time. Sometimes it's not even possible to remove the noise.

The ideal solution would be to implement an automatic texture "mipmapping" system, where we used different texture resolutions for sampling based on ray differentials. We must aim for that solution, but it will take quite some time.

Therefore, for a short term "quick fix" I've implemented a new functionality that allows to use the original texture for objects near the camera and just the average color of the texture for far objects. In the middle, a "blend" will be progressively done between the original texture and the averaged color.

This is FAR from a good solution but it might solve some of the current noise problems.

The way this works is:
* Parameter to enable/disable this function.
* Parameter to set the minimum distance. Any surfaces nearer than this distance will show the original texture
* Parameter to set the maximum distance. Surfaces farther away than the maximum distance will only show the averaged color. Between min and max, a progressive "blend" will be done between the original texture and the averaged color

#15

Hi David,

we have accurately reviewed your "fix" for the mipmap problem, but for our needs, it isn't a good solution.

With that fix we must implement many modifications to our code, after which, that the result seem to be very far from what we need. In particular we need to implement the following changes:

Considering that the distance parameter is by texture, we have to group our objects by distance and if there are objects with the same texture, we need to replicate the texture definition. For a simple model a duplicated copy is insignificant , but for a large model, where in average we need almost 8 mipmap levels, every texture definition will is therefore replicated many times. Furthermore, when calculating the right distance beetween the objects in order to have good results, it becomes quite tricky and complicated and is far from a quick modification in our side.

Having said this, because this is a fix that doesn't solve the problem completely , for the time being, we prefer to continue without the fix and wait for the final solution.

 

Best regards,

John.

#16

Hello, John.

Thanks for the info. It's dissappointing but not really unexpected.

I've removed the Distance Averaging feature from the code now, as it's adding complexity to the code and is not useful at all as you explained.

I'm investigating how to implement mipmaps in YafaRay properly, but it will take some time.

Thanks and best regards!

#17

Priority:normal» critical
Status:needs work» needs review

Hello,

Implementing mipmaps is not going to be easy. I have to check the current (existing but unused) Ray Differential code to see if it works as expected. Afterwards I have to decide the best way to implement mipmaps taking into account:

a) speed

b) memory usage (I expect a significant memory increase)

c) quality of result in different conditions (near, far, anisotropic texture mapping, oblique viewing angles, etc)

Of course, mipmapping would be enabled/disabled by a parameter, so even if this would increase memory usage/processing time the user would have control over it. Also, I expect the additional processing time will be more than compensated by the reduction in the necessary AA samples to reduce the noise from the textures suffering aliasing due to lack of mipmapping.

There are several possible ways, but if we implement this feature I think it would be better to go for the implementation that gives the best possible quality, if possible.

 

So, I'm studying this paper: https://graphics.stanford.edu/papers/trd/trd_jpg.pdf  to see if I can implement it in YafaRay.

Also, there is an alternative to mipmaps called "ripmaps" or anisotropic mipmaps, as explained here: https://en.wikipedia.org/wiki/Anisotropic_filtering

 

At this stage it would be very useful to get all possible opinions and feedback from all of you, guys. Please let me know what you think should be the best way forward.

 

Best regards. David.

 

 

#18

Hello David,

we trust in Your work and commitment to bringing YafaRay to higher and higher levels.

Indeed, to eliminate the noise we raised the number of passes (AA Sample, etc.), but in these cases processing time becomes prohibitive. We think Mipmaping will decrease processing time and overall quality will definitely be much better !!!!!!

We aren't experts in ray- tracing, so our help can only be marginal, however I have found these documents that seem interesting:

http://www.cs.rpi.edu/~cutler/classes/advancedgraphics/S08/lectures/18_s...

https://graphics.ethz.ch/teaching/former/vc_master_06/Downloads/Mipmaps_...

https://graphics.stanford.edu/papers/trd/trd_jpg.pdf

http://www.flipcode.com/archives/Raytracing_Topics_Techniques-Part_6_Tex...

http://iquilezles.org/www/articles/filtering/filtering.htm

http://www.pmavridis.com/data/I3D11_EllipticalFiltering.pdf

I hope to have been of help

John

ACCA Software

#19

Status:needs review» needs work

Thank you, John. I'm (slowly) progressing.

#20

Status:needs work» ready to commit

Hello,

 

For your information, I'm slowly progressing with MipMap creation & runtime detection code. This is being quite difficult for me but in my latest tests it seems it's kind of working.

So far, I've only implemented the most rudimentary MipMap interpolation type (trilinear) which is known to cause some artifacts, but it's the fastest mipmap interpolation. I plan to implement better (but slower) mipmap interpolation algorithms such as EWA in the future, but for now this is what I got:

Example without mipmaps (1 AA sample)

 

Example with mipmaps-trilinear (1 AA sample)

 

For now this will only work with textures mapped to UV and only when the meshes have UV maps associated with them.

I hope you find it interesting. Best regards and have a Very Merry Christmas!! :-)

#21

Hello David

thanks for the good news that you give us.

I will try to test the version of these days if I can, and I'll let you know !!!

In a few days we will close for the Christmas Holidays and will return to work Jan. 9, 2017, I do not know if I can try the version also because we are committed to move to our new headquarters, in any case before the end of the year we exchange greetings.

bye, see you soon

john

ACCA Software