A robotic surgeon with tiny lights, tiny cameras, and steady hands sounds like a miracle of technology. But what do the results show?

Even before laparoscopic surgery took off around 1990, several companies, backed by U.S. defense grants, were at work on robotic surgical systems.

Laparoscopic surgery has proven to be a significant medical advance, turning major surgeries that left scars and kept patients in the hospital for several days, into fairly minor procedures.

As robotic surgical systems moved through research and testing, many doctors hoped the new technology would increase those advances.

The companies building surgical robots were certainly optimistic. In product names like Zeus, Aesop, and da Vinci, one can hear great aspirations.

Zeus and Aesop were both purchased by Silicon Valley manufacturer Intuitive Surgical, and dissolved. So the hope that robotic-assisted surgery advances hangs on da Vinci, which was first approved for clinical use by the Food and Drug Administration (FDA) in 2000.

For the 2016 fiscal year, Intuitive Surgical reported revenue of $670 million, beating investor expectations. The company also told Fortune magazine that in July “the number of procedures done with a da Vinci system jumped by 16 percent in the second quarter compared to a year earlier.”

The system doesn’t resemble a robot so much as a video game. A surgeon sits behind a screen and looks at a magnified view of the surgical site while operating the machine’s robotic arms.

The robotic arms can get into hard-to-reach places, promising patients less bleeding, faster recovery, less chance of damage to important nerves, and smaller scars than traditional surgeries.

A single robot costs about $2 million. Some of the attachments that go on the arms are disposable. And robotic surgery generally costs anywhere from $3,000 to $6,000 more than traditional laparoscopic surgery.

So is this the brave new world of medicine or an expensive, ineffective technology?

One thing is certain: The da Vinci hasn’t improved patient outcomes as dramatically as the first wave of minimally invasive surgery did.

A decade into its use, the laparoscope had proven that patients fared better with its smaller incisions than they did with “open” surgeries, or those that required a large incision.

“As laparoscopic surgery has continued to succeed, I don’t think there’s a person on the planet who would have an open operation. And that’s only over 20 years or so, so that’s a rapid shift,” said Dr. Jay Redan, the president of the board of trustees of the Society of Laparoendoscopic Surgeons, and a charter member of the Society of Robotic Surgery.

Fifteen years into use of the da Vinci system, evidence that it trumps other methods is lacking.

The ECRI Institute, a nonprofit organization that synthesizes data on medical procedures, drugs, and devices to support hospitals and doctors in creating quality protocols, has analyzed more than 4,000 studies on robotic surgery.

“The evidence isn’t strong enough to determine whether or not a robot is better than traditional minimally invasive surgery, but the evidence does indicate that it’s better compared to open surgery — more evidence from higher quality studies may change this conclusion,” said Chris Schabowsky, Ph.D., a program manager at ECRI.

To justify its price — roughly 10 times that of a traditional laparoscopic surgery — da Vinci would need to do a lot better overall.

“This is a technology that is costing the healthcare system hundreds of millions of dollars and has been marketed as a miracle — and it’s not,” said Dr. John Santa, medical director at Consumer Reports Health. “It’s a fancier way of doing what we’ve always been able to do.”

Da Vinci was originally designed to do cardiovascular surgery, but it’s fallen out of favor for heart surgeries. Next it was picked up for gynecological surgeries. In 2013, the American College of Obstetricians and Gynecologists (ACOG) said it wasn’t the best, or even the second-best option, for noncancerous gynecological surgeries.

Researchers at Columbia University published a study that showed da Vinci costs $3,000 more than a traditional laparoscopic surgery to remove an ovarian cyst.

Some critics called da Vinci a “solution in search of a problem.”

Da Vinci was finally acknowledged for use in urology. Prostate removals were difficult to do laparoscopically, and many surgeons continued to use open incisions. The da Vinci made it easier to do minimally invasive prostatectomies. Nearly 90 percent of these surgeries are now done robotically.

Patients who undergo robotic prostatectomies lose less blood, but in the measurements that count most — how likely they are to be impotent or incontinent after surgery — the robot is no better than open surgery.

This was confirmed in a report published in July in the medical journal The Lancet. The study — the first of its kind — assessed the initial stage of a two-year trial of robotic-assisted surgery vs. nonrobotic surgery for prostate cancer.

Roughly 308 men with prostate cancer were part of the study. About half received robotic-assisted surgery and the other received traditional open surgery. A follow-up after 12 weeks looked at urinary and sexual function and saw no difference in outcomes.

The only disparity involved recovery. The men who received the robotic-assisted surgery spent less time in the hospital.

The only area where robotic-assisted surgery may have the upper hand is in prostate cancer treatment after the procedure. A study published in European Urology showed that prostate cancer patients who had “robotic-assisted surgeries have fewer instances of cancer cells at the edge of their surgical specimen.”

This could make it less likely that those patients will need additional cancer treatments such as hormone or radiation therapy than patients who undergo traditional open surgeries, according to press release from UCLA.

However, experts interviewed by Healthline blame negative outcomes from robotic-assisted surgeries on the surgeons and not the robots.

“When [laparoscopic surgery] was introduced, there was a spike in patient complications. That was because, in general, the surgical field was getting trained. There were errors, there were mistakes. Now fast-forward, this is just kind of par for course when it comes to introducing a disruptor. You’re going to run into these issues,” said Schabowsky.

For patients, the key to minimizing the risk of complications is to ensure their surgeons have ample experience with any device they’ll use in the operating room.

However, information on surgeon training on robotic systems can be hard to get, the experts agreed. Only cardiac surgeons currently make that information available to the public.

In general, experts say physicians in general need to complete 20 to 30 robotic-assisted procedures before they can be considered adequately trained.

“Americans tend to think that the latest and greatest technology has got to be better, and it’s not in this case,” said Santa.

Intuitive has undertaken direct-to-consumer marketing for its robots. As a result, patients often demand robotic-assisted surgery.

“I can’t tell you how many patients come in who say, ‘I want robotic surgery with a laser’ — and they’ll find somebody to do that,” said Dr. Eric M. Genden, an ear, nose, and throat surgeon at the Mount Sinai Hospital in New York. “This is a beautiful illustration of how American medicine and the patients tend to become enamored of technology without ever really asking the question, ‘What are we getting for the technology?’”

Redan agreed.

“Patients will go to someone who has the robot because it’s been marketed so much,” he said.

Hospitals advertise their da Vinci machines in part as a response to perceived consumer demand. They see the robots as a way to bring more patients through their glass doors rather than their competitors’, studies have shown.

The market push appears to be working.

“Within five years, one in three U.S. surgeries — more than double current levels — is expected to be performed with robotic systems,” according to Fortune magazine.

And it’s not just urban hospitals that are taking the plunge.

The Wall Street Journal reported in 2010 that 131 hospitals that installed da Vinci systems had 200 or fewer beds. Overall, roughly 1,500 U.S. hospitals have installed the da Vinci Surgical System since it came to market in 2000, according to Modern Healthcare.

In turn, hospital advertisements also help drive the perception that robots make the best surgeons.

One study of how hospitals talk about robotic-assisted surgery found that many copied directly from Intuitive’s marketing materials. A minority of these hospitals pointed to potential risks. Unlike doctors and drug companies, hospitals aren’t required to disclose risks in their advertising.

Marketing has played such a big part in drumming up demand for da Vinci machines that one surgeon who has developed a substantial online following under the pen name Skeptical Scalpel concluded in a blog post on robotic-assisted surgery that, “The decline of medicine as a profession began when it became legal for doctors and hospitals to advertise.”

Dr. Fabrizio Michelassi, the chair of the department of surgery at Weill Cornell Medical Center in New York and chair of the board of governors of the American College of Surgeons, said surgeons are obligated to educate their patients on what the evidence says are the pros and cons of robotic surgery.

“Unless we inform the patient population on this, there will be a drive from the consumer that trumps everything else, because at that point hospitals and physicians are caught in a difficult dilemma,” Michelassi said. “Hospitals and physicians are caught in the dilemma to either continue to deliver optimal care or to respond to market requests.”

Related News: Men Still Overtreated for Prostate Cancer »

The irony is that hospitals lose money on robotic-assisted surgeries because insurance companies reimburse all minimally invasive surgeries, whether laparoscopic or robotic, at the same rates.

However, hospitals in rural areas that are designated as Critical Access Hospitals (CMAs) by the Centers for Medicare & Medicaid Services are at an advantage because of federal reimbursement rules for such facilities.

A 25-bed hospital in rural Wyoming told Modern Healthcare that it expects to recoup 40 percent of the cost for the da Vinci purchase because of its CMA status.

Dr. Richard Newman, a pancreatic and endocrine surgeon at Saint Francis Medical Group in Hartford, Connecticut, researched the cost effectiveness of the da Vinci by pairing cases of gallbladder removal with identical outcomes, one laparoscopic, one robotic. He found that the robotic-assisted surgeries cost three times more.

“I don’t think it’s a good deal for the hospitals,” he said. “The administrators that are in place have been in a very volume-driven field, where if the competition across town gets one, you get one.”

Hospitals do seem to recoup the cost of the robots through volume. One way is by using the machine as much as possible.

Investment experts also told Modern Healthcare that in order to make the purchase of a da Vinci Surgical System feasible, hospitals must perform anywhere from 150 to 310 procedures within six years to offset upfront and ongoing costs.

Several doctors confirmed that hospital administrators, who are the gatekeepers for their operating rooms, staff, and equipment, are inclined to okay robotic-assisted surgeries to defray the cost of the multimillion-dollar machines.

The ECRI Institute ranks robotic-assisted surgery among its top 10 healthcare hazards for 2015. ECRI doesn’t fault the device. Instead, it points to inadequate certification requirements at the hospitals that use it. The group is pushing hospitals to develop appropriate processes for approving doctors to use the robotic systems.

Some hospitals may require surgeons to perform three robotic surgeries before giving them the okay to operate on a patient with a robot. Others may require 50 or 100 operations. Hospital policies are not routinely disclosed to the public.

“You do three robotic cases and you’re credentialed,” Redan said, by way of example. “But people spend a year in their fellowships learning how to do conventional surgeries.”

In some lawsuits pending against Intuitive, some plaintiffs claim that the company lobbies hospitals to ease their credentialing requirements to allow more doctors to use the machines.

To protect its assets, in 2014 Intuitive “took a pretax loss of $77 million to resolve the estimated cost of product liability claims,” according to the San Jose Mercury News.

In July, Intuitive settled “a lawsuit brought by a Placer County [California] woman who blamed severe internal injuries” due to “a hysterectomy seven years ago on an early generation of the Sunnyvale-based company’s robotic arms,” according to the newspaper.

Although the final terms were confidential, the plaintiff sought $10 million in damages.

Intuitive has expanded the training it offers doctors to get them started. But many still think it’s not enough. Although device manufacturers aren’t required to train doctors on their equipment, experts say Intuitive has done more to push patient demand than to train surgeons.

“Intuitive are probably the worst at doing this and probably the most responsible. Their direct-to-consumer marketing is just criminal. Their lack of training, in my opinion it borders on criminal,” said Genden.

Intuitive has also continued to push into new surgical departments, making the case most recently that da Vinci works well for head and neck surgery and even the open, and usually relatively low-cost, category of general surgery.

For example, based on his experience using the da Vinci, Genden said it had been a boon for transoral surgeries to remove tumors in the throat, shaving hours off the operating time — which correlates with lower patient risk, Genden said.

But rather than stop there, Intuitive has pushed for the da Vinci to be used for thyroid removal. Although a thyroidectomy is generally done with an open incision, the robotic procedure took significantly longer and didn’t give better results, Genden added.

The Mount Sinai Hospital does not offer robotic-assisted thyroidectomies. But here’s what Intuitive had to say about the procedure in its 2013 annual report: “Open surgery is an effective surgery in terms of oncologic control and has low complication rates. However, it leaves a prominent neck scar. Surgeons, predominantly in Asia, are now using the da Vinci Surgical System to perform thyroidectomies entering the body from the axilla in order to avoid the visible scar on the neck.”

Intuitive is hardly unique in its effort to find new uses for its products, said Santa.

“It highlights another problem in our system, which oftentimes nobody’s happy confining something to a fairly narrow corridor. They want to make as much money as they can off of it,” he said. “We see this with drugs, we see this with devices, that if it works for A, B, and C, let’s try it for D.”

Da Vinci is both a cause and a symptom of a U.S. healthcare system that costs far more than comparable systems in other countries without offering better results.

To help hospitals determine if a robotic surgical device is the best purchasing decision, the ECRI Institute developed a free assessment tool.

Through Robotic Surgery Planning, hospitals can evaluate core components like practicality, patient safety, quality, and cost.

“We developed this service to help hospitals decide if this costly mode of surgery — which currently has limited clinical evidence and the potential for overuse — is appropriate for their needs,” said Thomas E. Skorup, M.B.A., F.A.C.H.E., vice president, applied solutions group, ECRI Institute, on the company website.

There’s some evidence that among consumers who may see surgery under the care of an unerring robot as less scary, and hospitals, pressured to recoup the costs of the robot, the result may be surgeries that aren’t entirely necessary.

As robotic-assisted surgery has become the most common way to do prostatectomies, the number of these surgeries has risen against a backdrop of medical guidance that increasingly identifies the best way to treat prostate cancer as “watch and wait.”

Dr. Quoc-Dien Trinh, a Harvard urologist who uses the da Vinci, was reluctant to conclude that people who don’t need surgery are getting surgery. But the data points in that direction.

“It’s hard to incriminate the individual, but if you look at general trends in the population, that’s what it shows. These new technologies have always disseminated mostly in low-risk populations,” he said.

Redan has an idea of how to rein it all in.

“Currently, I think robots should just be used in certain centers of excellence where they are evaluating the efficacy, efficiency, and economics of this type of surgery,” he said.

Related News: U.S. Healthcare Is the Worst Among Developed Countries »

There’s another possible hero in 21st century medicine: The Affordable Care Act (ACA).

The law establishes a new compensation model, called Accountable Care Organizations, that rewards doctors and hospitals for good results. Part of that effort includes pushing Medicare-certified hospitals to publicize information on how their patients fare. Many hope that medical professional groups will follow suit.

That information would let patients have more informed conversations about whether to undergo robotic-assisted surgery.

Accountable care is a radical shift from the status quo, in which doctors are paid based on the volume of care they provide. Few hospitals are sure exactly how the new system will work. But those hospitals are increasingly evaluating care in terms of value, defined as the quality of the medical outcome divided by the cost of treatment.

That approach will put the squeeze on high-cost procedures, like robotic-assisted surgery, that don’t demonstrate any big advantages over cheaper alternatives.

“If you do the same equation of outcome divided by cost, you’ve got to get a really better outcome to justify the cost, because the cost is immense,” said Michelassi.

Of course, it’s also possible that technological innovation will prevail in the end. The da Vinci technology could improve, or there could be a new innovation that will deliver on its promises to send patients home sooner with better long-term results.

“I remember when we started to perform laparoscopic surgery, the instrumentation was very rough. Now, 25 years later, there is no question that some operations are easier laparoscopically than open, but for a while then every operation was more difficult laparoscopically. Maybe the da Vinci platform will continue to evolve to a point of becoming much more affordable or offering major advantages over other surgical approaches,” Michelassi said.

We’re just not there yet.

Editor’s Note: This story was originally published on February 12, 2015 and was updated by Carolyn Abate on August 10, 2016.