Depth Sensor Alignment

I have just recently received my POP2 and have been going through the “getting to know you” phase with the scanner. As I have been doing my testing I noticed something which I wanted to validate with Revopoint, other users, to understand if it may be a factor in some of the issues I have been experiencing. I’ll start by saying I have quite a few 3D scanners in my arsenal ranging from standard light projection, to IR, to LiDAR standalone (Intel L515) and integrated units (ala iPad PRO), so I am very familiar with the trials and tribulations involved with 3D scanning as well as the strengths and weaknesses of the various technologies employed. As I have been testing the POP2 I have been having quite a bit of trouble with both obtaining and maintaining tracking in what should be ideal conditions. I am not yet ready to say there is a problem with my unit, however, as the POP2 is new to me, and even with the other experience each device is unique. I did notice something, however, that if it is not an expected/normal state, could be directly responsible or at least exacerbating my general scanning and tracking issues. This post is to put it out there, and see if others have noticed it, or at very least find out if it’s a normal state for this device .

The Issue:
I first noticed this on the iPads PRO connection, what I describe below seems to be a bit more noticeable in the mobile app for some reason, possibly just because of the extremely bright/crisp screen. While observing the depth sensor feed I started to notice that 1. there is a pretty serious amount of parallax between the depth and the RGB camera views. They are far enough apart that I expect this some, but they also appear to be operating at different focal lengths. It’s entirely possible that this is normal, but I wanted to note it as it did seem a little more extreme than I expected. 2. As I see data move over the depth sensor, I noticed that the feed appears to be coming in a bit cockeyed as if it were not mounted straight. The first issue I have seen before, at least in part in other devices; this, however, I have not, and as such is what I am most interested in validating to be an issue, or not an issue, so that I do not tread down pointless troubleshooting paths. Other than describing it as cockeyed, or tilted, I think a screenshot will best illustrate. It’s possible I’m seeing a cropped part of a very wide angle lens view (say the NW corner) which might make it appear slanted from that perspective, but without the ability to do a calibration I don’t really have a way to tell for sure if that’s the case or not.

So, next time you scan something, watch your depth feed when on a particularly good full-screen depth hit, and please let me know if you guys are seeing this same behavior below - and if you are seeing it, whether or not you have been able to maintain a reasonable quality of scan and subject tracking. Note that the areas I point out in the screen shot NEVER see any depth hits and remain blank .

Screen Shot:
image

Thanks all!

The cameras has nothing to do with the scan results , the “Depth Camera” has a different focal length than the RGB but it is only for visualization , but what is really tilted is the Infrared Laser that project the slightly tilted pattern on your object as that is what is important and read by the software .

Issue with tracking ? simple too many points with too smooth features , how more accuracy how difficult it is to track , recommend using short sessions and not bigger than 200-300 frames if not using trackers . The turntable that came with POP2 is very wobbly , only good for a very small objects , I am using my own tournable with adjusted speed .

Try to get recommend setting for each mode , and believe me when I say that scanning objects with lower accuracy is much easier , but simple surface objects are not … so low accuracy for smoother objects with less features, and higher accuracy for complex objects .

I did not experienced any issues with my complex or non complex scanning , the only one issue I have is when I start scanning the Auto setting for the RGB camera jump back to manual, this bother me as I have hard time to get proper exposure of the textures. On Manual it simple do not works for me for some reason as that should be my first choice .


I don’t believe the tilt matters, as the position of your POP2 while scanning do not influence the scanning result unless the already calibrated MODES ( face, body, feature) do not correspond with the position of the Infrared Sensor

Since the software is still in beta , anything is still possible , but you can’t change the tilt of the infrared laser, you can only adjust the software to its position by calibrating it on different focal length , small drop of a package could influence it already , lucky me I have no issues at all .

Also make sure you adjust proper your Depth Camera exposure … it is crucial in tracking , if it is too bright you gonna lose the tracking in second.

1 Like

@PUTV thank you for sharing your screens with me as well, it is very helpful. I want to be clear about what I was attempting to demonstrate above, though. I am talking about the edge to edge return of depth data on depth camera screen. I can see from your screen shots that you are seeing data edge to edge though it looks like a tilt difference in the subject view. I am not really concerned about that as I know calibration can resolve such things, however, I am not speaking of the RGB texture mapping/calibration at all here, I am talking only about the depth sensor. The view that I showed above was about as close as I could get to illustrating depth detection over the entire frame. The tilted black “frame” that I am pointing out at the far edge of the screen are basically dead spots. No matter how optimal the subject material, distance, or detail level that area of the depth camera never displays any depth data at all. The closest thing I can think of would be to compare it to the “letterbox” view of wide screen video displayed on a 4:3 format display, though obviously not symmetrical or purposeful as in that example. The main reason that this behavior raised a red flag for me, is because that issue, in addition to the behavior of the tracker, both are very similar to what I experienced on another of my IR scanners when it started to go out of depth sensor calibration (not RGB alignment calibration).

All of the things you mentioned regarding subject, detail level, and depth exposure have not really had any demonstrable impact on my tracking results (it has been losing tracking even in good environmental and subject conditions without anything moving at all - it is almost like it is trying to track a ghost image/being spastic, which is again something I have seen when alignment is seriously off). Keeping the sessions short is not really an issue as I have to abort generally before I hit the first threshold anyway. While it feels like past experience with depth calibration problems on similar devices, I do not want to anchor myself in faulty diagnosis or jump to conclusions, however I have to follow Occam’s Razor and do what I can to remove the variable given all of the other things I have ruled out. I have been too busy to keep up on the threads but I am hoping that there is a calibration utility available now for the POP2; I do not wish to waste any more time troubleshooting before I am able to do at least that and mitigate it as a variable. I did also notice that my box was a bit “roughed up” when it arrived, so I am not ruling out that it might have taken a bit more of a tumble than it could absorb without having a negative impact on it somewhere in the shipping/delivery process. :frowning:

@EventHorizon i just calibrated my POP2 yesterday and the infrared patterns now projecting vertically independently from camera position what is new . It is already good but could be better

There are couple of major bugs in the new software that causes most of the issues that people experience. I hope they address it ASAP other way we gonna waste time on talking without proper solution .

  1. Gain control of Depth sensor ( tracking )
  2. Exposure control of RGB sensor ( textures )

The visual preview do not corespondents with actual settings or provide any adjusting . Right now it has mind on its own . Starting the new scan switch it back and forth as it please .

I am both happy and sad by that news, I feel so conflicted! :wink: Now that I managed to get the calibration app to install I will be giving it a go this evening. I’ll report back in case anyone has a morbid sense of curiosity. :slight_smile:

2 Likes

Definitely , please take you time , the second part of the calibration can be very frustrating , please do not move the target while calibrating … I have to do it twice … and most important please save your old calibration files just in case it get worse … just to be safe

my last results improved from 0.0700 +

1 Like

Thanks for the info! I finally had some time to run the calibration. Thanks to the warning I had no issues moving the calibration pad, however I have been unable to get a better result than .17. I will give it another go tomorrow and see if I can improve upon that. I noticed that several times the device would be showing me very much off of horizontal when I was clearly not (or at least not nearly as badly as shown). Other times it would seem to be stuck in a lean and no movement either way would bring it in (while it was complaining on screen “Horizontal!”. Did you have this problem at all? And during the 2nd and 3rd phase tests, was it your experience that the on screen dots didn’t align at all with the on camera display? The video shows the example person keeping horizontal and not adjusting to meet the tilt of the board, and as such I expect some perspective shift… but they seemed really quite off in these steps (though it would go green once I got at least approximately in place). I want to make sure I am following the process as intended and not over correcting for anything, but the above two things have given me pause as well. Will report back tomorrows attempt, maybe I wasn’t seeing straight and was shaky when trying this last time… :slight_smile:

Not so good results , I tried 2 times the first time was bad , I almost given up on the second phase as same as you I had issue with the proper horizontal alignment to match the dots , it was just Impossible , but somehow I made it . After 3 days of intense usage I noticed the value starting to grow up again , I was on 0.08 today from around 0.05 after calibration , the heat may change the accuracy , I will test it out today when the device is still cold and see the new result .
I believe it is going to be better the second time for you , 0.17 is much too high for a new device , you should get at least 0.07 or lower but I never got below 0.05 , also make sure your table is leveled for less issue with the alignment vs the horizontal position of the device and steady hands . I was thinking to use tripod for that and keep perfect horizontal position , it may helps looking forward to see your new score.

Hey there @PUTV, sounds like you have had your own ups and downs! I will say this, I will add some empirical evidence to your theory around heat bringing down accuracy. Another of my IR based scanners has the same issue - when it gets heated up from a long session it starts to fall apart on accuracy/tracking. I can’t speak for the POP2 yet however as I have not gotten it working well enough to bother trying any lengthy sessions. After quite a lot of testing my process and results:

My General Process:

  • I tested the calibration under various lighting scenarios, I wanted to make sure that environmental factors were not causing me trouble. While all the spots I tried were varied, they were all what I would consider to be REASONABLE locations to expect a calibration to do well.

  • I tried various holding techniques for the test from exactly as shown in the tutorial video to being highly braced on the ground for maximum body/arm stability. While the results did change some the variance was minimal. Through these many iterations I have become quite the expert at quickly getting through the calibration phases, almost going straight to green on each new position (except for the wonderful phase 2 from time to time…) I have continued to note that the “HORIZONTAL” error still happens every so often… it’s like the sensors get stuck or the like as it will show a tilted device, yet rolling the orientation does not budge it (and yes, while doing so slowly and waiting for any frame lag to catch up). I think that part of it may be that the feedback for transverse axis pitch issues vs longitudinal roll issues do not seem to be clearly differentiated which may generate some confusion on both sides. That said, the majority of my test calibrations I did not receive that error.

  • I decided to mitigate human shake in the process by rigging up a standing tripod and a boom mount. This allowed me to expand and retract the boom, and raise and lower the stand in answer to the calibration changes. (a dynamic microphone holster interestingly enough holds the standard handle for the POP2 quite nicely and securely). This worked well, and did in fact mitigate a lot of excess movement, but was a little awkward.

Summary of tests (25 calibrations run in total):

  • ALL of the above testing variations, styles, stabilizations unfortunately failed in any way to get me closer to a good alignment. Even with the wide swing in process above and what I thought were some VERY clean calibration runs, I was never able to get better than a .15 and most of them remained in the ..17 and .18 range, especially towards the end of a lot of tries as the sensor really warmed up.

Sadly I am not really sure how to proceed. I have tried everything that I can think of within reason, and I have to assume that it should NOT be this hard, especially on a new device. I am open to any suggestions. @Revopoint-Cassie if there are any other things that I should have tried please let me know as well, I am excited to really use the scanner, but until I can get this resolved it is unfortunately nothing more than a paper weight to me. :frowning: Again, any thoughts/suggestions are very welcome, it is frustrating because I have seen glimpses of what it can do, enough to be a believer, those glimpses thus far are frustratingly close, but out of grasp.

1 Like

Wow that is a lot of effort for this lover result . 0.15 will not prevent you from scanning as the maximum allowance is 0.5000 to be really worried about . I know @Revopoint-Cassie made her calibration and ended with something about 0.13 only .
Most of the calibration was loaded by default to the POP2 devices and not calibrated individually .

I don’t know what to say at this point , as I have zero new tips for you . I just wonder if all devices can be calibrated to the lowest values , as I never could pass below 0.0500 , and I did my calibration with a very little ambient light almost darkness .

@EventHorizon Hola te cuento mi experiencia por si fuera de ayuda para ti Recientemente estoy empezando en el mundo del 3D de manera autodidacta y aprendiendo de tutoriales y vídeos de la comunidad.

Cuando realicé la primera comprobación en el test de calibración arrojó un resultado de 0,16. Este valor era el que traía de fábrica el Pop2.

Pensé que mi pop2 estaba deteriorado porque cuando comencé a realizar la calibración y acercaba el escáner a 15 cm o menos se quedaba media pantalla en negro, pero parece que es normal.

Tras 4 calibraciones prácticamente sin éxito, solo conseguí bajar el valor a 0,1416.
También comprobé que la mesa estuviera nivelada.
La iluminación era la luz blanca no muy intensa de la lámpara del salón.

Al día siguiente probé cambiando el cable USB C por el cable 3.0 y con la luz natural que entraba por la ventana realice la calibración y para mi sorpresa obtuve un buen valor de 0,0499.

A continuación realice una nueva calibración y el valor final que obtuve fue de 0,0535

Hoy he realizado un escáner en modo rápido y oscuro de una escultura.
Tras realizar el proceso de escaneo hasta el stop, la previsualizacion parece que el escaneo ha salido muy mal pero tras realizar la fusión de puntos, la malla y la textura, el resultado ha sido muy bueno.

He probado con el modo marcador y parece que no funciona correctamente porque pierde el seguimiento continuamente.

Saludos

Thanks @WOLF, that’s very interesting to hear about your results, when you say you switched the USB C cable for the 3.0 cable, do you mean that you were using the mobile split cable (the one with the USB C and A on one end, and the camera connection on the other)? And then you switched to the PC connect cable which is just type A to the camera interface? When you made the change did you also notice a change in the latency of the video feed from the camera as well? It is an interesting thought; I am already using the latter cable, and unfortunately since the POP2 is using the old style of USB3 connector I do not have any high speed USB3 cables with that type to test with. (one of the reasons I wasn’t swapping cables as a matter of my normal troubleshooting). You make a very good point which reminds me of a flaw in my troubleshooting, so I really appreciate you bringing it up! I would also assume that both the 2 in 1 mobile cable and the A to C cable are USB3 compatible so I am going to try first swapping to that cable just to see if it makes a difference. I will also see about securing a new and proven high speed USB3 cable with the old connector (they are very hard to find here any more, but I am sure I can find one). I DID at least try multiple ports on multiple computers to rule out the port being an issue. I will give this a go and report back so wish me luck!

1 Like

@EventHorizon El cable que use al principio fue este;


Y luego este;

Mucha suerte amigo.

@PUTV yeah, but a reference of .5 deviation would be a travesty for a device that is supposed to have a .1mm level of accuracy. To be honest anything above a .1 average variation on a .1mm device would be unfortunate in my opinion. @WOLF shared his experience with a change of cable, which is the ONE thing I was not really able to do since I do not have any high speed USB3 cables with that old style of connector, though I did test on multiple ports to make sure that end was not a problem. Thanks to his reminder, though, I am going to secure a known good USB3 cable of that connector type. I also forgot to use diagnostics to check the link type established, so I am going to verify the link etc. with a USB monitor. If you have never used it, this is an amazing tool for troubleshooting USB connection issues/devices. I can’t believe I forgot to use it as well: View any installed/connected USB device on your system (nirsoft.net)

I will give the cable side a go and see if I am able to make any progress; I will report back, wish me luck! :slight_smile:

@PUTV Muy interesante no lo conocía, muchas gracias.

Aha thank you for clarifying; mine did not come with the USB C to B cable, it came with a USB type A to B and then a split shorter cable USB type A and C to type B. Either way it’s at least another cable to test, so wish me luck!

Oops scratch that, I forgot how short the split cable is, no way I’m going to be able to calibrate with it. Off to find another cable it is!

I am using USB C , maybe that why

@EventHorizon I am also using my Nikon UC-E14 USB 3.0 sometimes as well as it can be much faster than the original , sadly my POP 2 died today on me , so the end if scanning for now . it was a short adventure . Edit: My POP2 restored itself again

Ouch, so sorry to hear your sensor died! Any resolution? (no pun intended)

On my end, I have been busy but was able to do some more testing. I have now I believe exhausted all reasonable avenues. I have tried several new cables, just to see if it could have been part of the issue, however that did not change. I have tried machines ranging from 8700K machines to 12900k and even a 5900X AMD machine. I have tried stock USB 3.0 ports to PCIE based USB 3.2 Gen2x2 20gb ports. Unfortunately the device just seems unstable - calibration is wildly all over the place, I have had complete calibration failure show at the end even after completing all the tests in green. The rest of the tests that do work run between 1.5 and 2.5 on average. Now, even if it were to be able to scan reasonably at 1.5 variation compensation, the issue is that it just goes unstable. The same issue in calibration as when scanning, I can have the device hovering over the calibration dots perfectly, and it will flash green a moment, then flash saying to match with the BIG dots, then back to green, etc. I can back the sensor off and bring it in and it will do the same thing. Holding still even, it will blip in and out of green good, and the message about the dots. On the scanning side it’s the same… when it does lock it will work for a bit, but then lose tracking never to regain it, spinning the subject off into the void with a trail of textures. (using the rotator). At other times, I can simply start a static scan with neither sensor nor subject moving, and it will bounce around and off as if there were a LOT of movement. Breaking the scanning sessions into small small bits might work for one revolution of a subject, but by the 2nd or 3rd it’s pretty much guaranteed that one of the above will happen and ruin the scan. And keep in mind this is all scanning in good environments and with scanning friendly subjects (no dark, shiny, etc.). And unfortunately whatever is going on with it, seems to be getting worse, as before I was able to get down to a 1.2 variance… ONCE… and I have not been writing all of these failed calibrations to it. I even wrote the factory calibration back to the device just to be sure I didn’t have something go wrong very early on. Now I am lucky to get a 1.7 and have hit never before seen 2.0s and complete calibration failure a few times. Part of me wonders if the failure could be more with the gyroscope/velocity sensors and not the depth… I don’t know how much of a factor they play in alignment, but I do notice that the orientation feedback during calibration can be REALLY off for me.

I wonder if there is a way to calibrate just the gyroscope/velocity side of things (if it does in fact have any influence on tracking)