Very simply, for example:
- On the background / the substrate on which the model and that will take a shadow, apply a material VRayMtlWrapper .
- In its slot Base material to enclose the necessary background material.
- In the Matte properties tick Matte surface and set the Alpha contribution -1.
The same can be done by selecting the same object and the background by clicking on it the right mouse button. The context menu appears. There choose V-Ray properties and it is necessary to install the same as in the above process the material. In fairness, it should be noted that the shown you render it a complete substantive 3d rendering , no "cheating" with the background.
What are you trying to get this Shadow and Reflection rendering elements, and both that and that with his alpha, but still mixed together in one render layer😁 VrayMtlWrapper works well only with the Shadow. With Reflection his problems. Whether so intended, whether it fixes, but the situation is such. That is, as the splint can get out of this situation: Matte shadow and reflection using V-Ray 2.0
But as for me, it's easier to render the desired render the elements and in the same photoshop them zakompouzit receiving the desired image.
Composite materials V-Ray does not work with standard materials of 3ds Max. For everything to work correctly, you must only use materials V-Ray. Standard material with self-illumination can be replaced either VRayMtl with a non-black color «Self-illumination», either literally luminous VRayLightMtl . It all depends on what you want to achieve from the self-luminous mat.