Sign in

General manager of analytics, data science and data virtualization at TIBCO. Former CEO of StreamBase. Medium author on analytics, parenting and cancer.

Volkswagen invested five years and nearly $50 billion into their long-awaited answer to Tesla. ID.3, their flagship electric vehicle, rolled off the assembly line. It didn’t work.

The car was functional, but VW’s programmers hadn’t yet figured out how to update the car’s software remotely. Owners reported hundreds of bugs. It wasn’t the fault of the programmers; VW leadership hadn’t paid attention to culture.

Digital transformation is about mindset and technology — connected cars are more like mobile apps than cars.

The ID.3 story is a reminder that mindset, not technology, and not money, matters most.

When you do something new, mindset matters most.

How to be a data science entrepreneur inside, or intra, a company that’s stuck in its old-school ways.

Mark Palmer,

Newsflash: you don’t work at a hyper-innovative company like Netflix, Google, or Amazon. But if you’re an entrepreneur at heart, can you thrive in a company that resists change? How is innovation in an established company different than in a startup?

Gifford Pinchot tackled these questions in his ground-breaking book, Intrapreneuring, in 1985. He studied big-company innovators of the day like AT&T, Du Pont, and 3M. The principles he uncovered have withstood the test of time.

If you’re a data scientist in a big company today, you’re an intrapreneur — an entrepreneur inside, or intra, a company. …

Seven points to consider in your collaborative decision-making process

Jeremy Perkins on Unsplash

The tech media is obsessed with data. But after eight years of measuring corporate data literacy, only 24% of companies report have reached data-driven nirvana. That’s fewer companies than last year.

Maybe being data-driven is the wrong goal.

Researchers Bart de Langhe and Stefano Puntoni think so. They advocate becoming decision-driven, not data-driven.

The distinction might seem small, but it’s not. It’s like communists versus capitalists, democrats versus republicans, or Red Sox versus Yankees fans.

Decision-making culture isn’t as cut-and-dry as saying Boston is a better sports town than New York. Data science is both creative and technical, like building…

Data science is taking off and failing at the same time. NewVantage Partners reports that 92% of companies are accelerating their investment in data science, up 40% year over year. Yet just 12% have deployed AI at scale — that’s down from 15%. (1)

So firms are investing more in data science and putting less of it into production. It’s like buying a Ferrari and leaving it in your garage out of fear.

What’s going on?

Sure, there’s cause to be careful with AI. Security, bias, and privacy, to name a few. But some IT teams go too far —…

Wind power forecasting is essential for sustainable energy production. But forecasts are made to be broken. Once in production, wind turbines adjust their position like a sailboat adjust to gather all the wind it can. Energy innovators are going real-time to seize this opportunity. They employ systems that learn and adapt forecasts to current conditions. Three technologies make it work.

Streaming data for wind energy comes from embedded sensors in turbines, streaming weather data, satellites, and drones. A single turbine alone has thousands of embedded sensors that emit data in real-time. …

Yesterday’s post, How AI Helps Find Electricity in the Wind, explored how data can help accelerate our national project for clean energy. Catalina Herrera, David Carr, and Geoffrey S. Lakings explained how data is essential for the greater good. But if you can’t visualize it, data is useless.

The Voronoi Polygon is one tool that can help. I think of it as the Where’s Waldo? of data visualization. They help humans make decisions that involve location, like the best location for a wind turbine. Watch this 55-second tutorial to see how they work.

Today’s #ModsMonday challenge is to create an…

There’s electricity in the wind. Gale force announcements by President Biden and GM have jolted the wind energy industry into high gear. Over the next decade, analysts anticipate a 3,000% increase in wind energy production. AI, data scientists, and human creativity will play an essential role.

So it’s timely that Catalina Hererra and a panel of wind energy experts describe the latest on how to use data science to optimize wind energy discovery and production. There’s so much in this session I’ve split it into a series of posts for this week.

Geoffrey S Lakings of Rippling Nature advises that…

It’s common for stories about AI to go down the slippery-Silicon-Valley-hype-slope with rocket boosters on. One author recently wrote:

“The productivity boosts from Artificial Intelligence (AI) and Machine Learning (ML) are the only prudent, intelligent and manageable route to maximize data’s value.”

This is like saying semicolons are the only path to great writing. Here are a few things about data that are more important.

Well-crafted visualizations that tell stories about data. Trillions are created every year and the great ones can change the world. Florence Nightingale’s Coxcomb chart, for example.

Curating new data is critical to support global warming…

How to democratize access to real-time data with Kafka for business users

Recently, this video about Kafka analytics drew a dramatic response ranging from accusations of spreading FUD (fear, uncertainty, and doubt) about KSQL to great interest from business analysts that see a need for something more. I found a voice of reason in Jesse Anderson in Why I recommend my clients NOT use KSQL and Kafka Streams, where he writes:

“Kafka isn’t a database. It is a great messaging system, but saying it is a database is a gross overstatement.”

Jesse does a nice job explaining why Kafka isn’t…

Uh, oh. I’m in charge: me, Jack, and Ruby in 2007. Jack looks a little worried.

Ten years of lessons after my wife died.

I became “both mom and dad” when my kids were six and four. My wife had just died, and I was panicked over the weight of my responsibility and then was relieved when a grief counselor told me that Jack and Ruby would be fine if I did my job. He said that, statistically, kids who survive loss could become better adjusted, more aware of the value of friends and family, and appreciate life more deeply, if I did my job. So learning my new job became my new obsession.

Over a decade later, as the kids head to college…

Mark Palmer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store