Data Engineering Podcast

Share on social media:

Support Data Engineering Podcast

[00:00:12] Unknown:

Hello, and welcome to the Data Engineering Podcast, the show about modern data management.   When you're ready to build your next pipeline or want to test out the projects you hear about on the show, you'll need somewhere to deploy it. So check out our friends over at Linode.   With 200 gigabit private networking, scalable shared block storage and a 40 gigabit public network, you've got everything you need to run a fast, reliable and bulletproof data platform.   If you need global distribution, they've got that covered too with worldwide data centers, including new ones in Toronto and Mumbai.   And for your machine learning workloads, they just announced dedicated CPU instances.  

Go to data engineering podcast.com/linode,   that's l I n o d e, today to get a $20 credit and launch a new server in under a minute. And don't forget to thank them for their continued support of this show.   And listen, I'm sure you work for a data driven company. Who doesn't these days? Does your company use Amazon Redshift?   Have you ever grown over slow queries or are just afraid that Amazon Redshift is going to fall over at some point? Well, you've got to talk to the folks over at intermix.i0.   They have built the missing Amazon Redshift console. It's an amazing analytics product for data engineers to find and rewrite slow queries, and it gives actionable recommendations to optimize data pipelines.   WeWork, Postmates, and Medium are just a few of their customers.  

Go to data engineering podcast.com/intermix   today and use promo code dep@signuptogeta50discount.   And you listen to this show to learn and stay up to date with what's happening in databases,   streaming platforms, big data, and everything else you need to know about modern data management.   For even more opportunities to meet, listen, and learn from your peers, you don't want to miss out on this year's conference season.   We have partnered with organizations such as O'Reilly Media, Dataversity,   Corinium Global Intelligence, and Data Council.   Upcoming events include the O'Reilly AI Conference, the Strata Data Conference, the combined events of the Data Architecture Summit in Graforum,   and Data Council in Barcelona.  

Go to data engineering podcast.com   / conferences to learn more about these and other events and take advantage of our partner discounts to save money when you register today. Your host is Tobias Macy, and today I'm interviewing Pete Soderling about his work to build and grow a community for data professionals with the data council conferences and meetups, as well as his experiences as an investor in data oriented companies.   And full disclosure that data council and Clubhouse   are both previous sponsors of the podcast,   and Clubhouse is 1 of Pete's companies that he's invested in. So, Pete, can you just start by introducing yourself?  



[00:02:44] Unknown:

Yeah. Thanks thanks for the opportunity to be here, Tobias.   I'm Pete Soderling, as you mentioned, and I'm a serial   entrepreneur.   I'm also a software engineer from the first Internet bubble, and I'm super passionate about making the world a better place for other developers.  

[00:02:59] Unknown:

And do you remember how you first got involved in the area of data management?  

[00:03:03] Unknown:

Yeah. I think,   funnily enough, the thing that jumps out at me is how excited I was when I was   a early developer, very young in my career.   I discovered this book, Database Design for Mere Mortals, and I think I read it over my my holiday vacation,   1 year, and I was sort of amazed at myself at how interesting   such a potentially dry topic could be. So that was a an early indicator.   I think fast forward,   my first company,   actually my second company in 2009   that I started was a company called Strata Security.   And originally we were building what we thought was a API firewall for web based APIs,   but it quickly morphed into   a platform that secured,   and offered premium data   from providers like Bloomberg or Garmin or companies that had lots of interesting proprietary datasets.  

And our vision was to become essentially like the electricity meter in between that data provider and their API   and the the consumers who are consuming that data through their API so that we could offer basically metered   billing based on how much data was consumed. So,   I guess that was my first significant,   interest as an entrepreneur   in the data space, back about 10 years   or so ago.  

[00:04:21] Unknown:

And now you have become an investor in data oriented companies. You've also started the series of conferences that were previously known as the DataEng Conf and have been rebranded as Data Council,   And that all started with your work in founding Hawka Labs as a community space for people working in the data engineering area.   And I'm curious what your original reason was for focusing your efforts in that direction and focusing particularly on data engineers.  

[00:04:51] Unknown:

Yeah. I guess it's it's gets to the core a bit of of who I am. And,   as I've looked back over my shoulder,   as both an engineer and a founder, I guess what I've realized, which actually to some extent surprised me is that all of the companies I've started have all had software engineers as end users or customers. And I discovered that I really am significantly passionate about getting developers together,   helping them share knowledge,   helping them with better tooling,   and essentially just making the world awesome for them. And it's become a core mission of everything I do. And   I think it basically is infused in in all these different opportunities that I'm pursuing now. For instance, 1 of my goals is to help a 1000 engineers start companies. And that gets to some of the startup stuff, which is essentially a side project that we can, we can talk about later. But specifically as it relates to Hacker   Labs, Hacker Labs was originally started in 2010   to become   a community   for software engineers. And originally we thought that we were solving the engineer recruiting problem.  

So we had various ideas and products that we tested,   surrounding introducing engineers to companies in a trusted way.   And originally that was   largely for all software engineers everywhere. And our plan was turn it into digital platform.   So it was gonna have social network dynamics where we were connecting engineers together   and those engineers would help each other find the best jobs.   So that was had, you know, very much was sort of in the social networking world.   But 1 of our marketing ideas was we wanted to build a series of events   surrounding the digital platform   so that we could essentially lead users from the the community in our events, and introduce them to the digital platform,   which was the main goal. And   1 of the niche areas that we wanted to focus on   was data because,   data science was becoming   a hot thing. Data engineering was even more nascent, but I sort of saw the writing on the wall and saw that data engineering was gonna be required,   to support all the interesting data science goodness that was being talked about and and really, was of interest to business. And so, you know, purely the the the data   meetups that we started,   were essentially a marketing channel for this other startup,   that I was working on.  

And ultimately that startup, didn't work and the product,   didn't succeed, which is often the case with network based products, especially in the digital world.   But I realized that we had built this brand   surrounding,   awesome content for software engineers, data engineers through our meetups that we had started.   And when I fell back on that community   and figured out that there must be another way to monetize and to keep the business going, because I was so passionate about working with engineers and continuing to build this, this community that we had seated And engineers love the brand. They love the events that we were running.   They loved our commitment to deeply technical content. And so 1 thing led to another and ultimately,   those meetups   grew into what Data Council is today.  



[00:07:49] Unknown:

And you mentioned that when you first began on this journey, it was in the 2010 time frame.   And as you referred to, data engineering as a discipline was not very widely recognized. And I'm wondering   how you have seen the overall evolution of that role and responsibility   and what the state of the   industry, and what the types of recognition and responsibilities were for data engineers at that time?  

[00:08:16] Unknown:

Yeah. I you know, data engineering was was just not really a thing at the time. And   only the largest,   tech companies, Google and Facebook,   even had the notion of sort of the data engineering concept. And but I guess, you know, what I've learned from from watching engineering   at those companies is that because of their scale, they discover things   more quickly and more often   or or earlier than than other folks, tend to. And so I think that was just a interesting, you know, leading indicator. And so I I felt like it was going to get bigger and bigger. But, yeah, there was no I don't even know if if Google necessarily had a data engineering   job title, at that time. So, you know, that that was just very early in in in the space. And, I think we've seen it develop a lot since, since then. And it's not just   in the title, but I think,   you know, we saw early on that data science was a thing and was gonna be a bigger thing. Data engineering was required to tee up the data scientists and the quants to do awesome stuff in the first place. And then there's also the analysts who are,   trying to consume,   the datasets   oftentimes in slightly different ways than the data science scientists.  

So I think early on, we saw that these 3 types of roles,   were super fundamental   and foundational to building   the future of data infrastructure   and business insights and data driven products. And so,   even though we started off running the data engineering meetup,   which I think we're still known for,   we pretty quickly,   through the conference embraced   these other 2 disciplines as well, because I think the interplay   of how these types of people work together inside organizations,   is where the really interesting   stuff is. And so, you know, as these these job descriptions,   the seams in these job descriptions and sort of how they unite and how they work together on projects is fascinating. And so through data council, our goal has been to further the industry by getting these folks in the same room around content that they all care about.  

And sometimes it's teaching a data engineer a little bit more about data science because that's what makes them stronger and better able to operate on a on a on a multifunctional team. And sometimes it's the data scientists getting a little bit better at some of the engineering disciplines and understanding   more of what the data engineers have to go through. So I think that,   looking at this world in a in a cohesive way,   especially across these 3 roles,   has really benefited us and made the community and event, very unique,   and strong. And now I should say that I think the next phase of that in terms of team organization,   and especially in terms of our vision with data council, is we're now embracing product managers into that group as well. I think that, you know, there's this stack. We sort of see the stack of data being,   data infrastructure on the bottom,   then data engineering and pipelines,   then data science and algorithms,   then data analytics on top of that. And finally, there's the the AI features and the people that are weaponizing this entire stack   into AI products and data driven features. And I think the final icing on the cake, if you will, is creating space for data oriented product managers. Because, you know, it used to be that maybe   you think of a data product manager as, like,   working for Oracle or being in charge of shipping a database.  

But that's, you know,   that's sort of a bit older school at this point, and there's all kinds of other interesting applications   of data infrastructure and data technologies that are not directly in the database world, where real world product managers in the modern era,   sort of need to understand how to interact with the stack and then how to build data tooling, whether it's internal tooling for developers,   or customer consumer facing   features. So I think,   embracing the product manager at the top of the stack has been super helpful for our community as well. And I find it interesting that you are bringing in the product manager because  

[00:12:12] Unknown:

there has long been a division in particularly with technical disciplines   where you have   historically the software engineer   who is at odds with the systems administrator,   and then recently the data scientist or data analyst who is at odds with the data engineer.   But there has been an increasing alignment across the business case   and less so in terms of the technical divisions. And I'm curious what your view is in terms of how the product manager   fits in that overall   shift in perspective and what their responsibility is within an organizational structure to   help with the overall alignment in terms of requirements and vision and product goals between those different technical disciplines.  



[00:12:59] Unknown:

Yeah. Well, hey. I think I think this is just a super this is a this question is a microcosm of what's really happening in the engineering world. Because   I think software engineers at the core, at the central,   location are actually eating   disciplines and roles that used to be sort of beneath them and above them. So again, I'm sort of thinking thinking in terms of this vertical stack. But, you know, most modern tech companies don't have DBAs   because the software engineer is now the DBA.   And, many companies don't have   designated infrastructure teams, because a software engineer is responsible,   for their own infrastructure.  

And some of that is because of cloud and other dynamics, but sort of what's happening is, you know, at its core, the engineer is eating the world, and it's bigger than just software in the world. The engineer is eating the world. And so I think the the absorption of some of these older roles   into what's now considered core software engineering has happened below, and I think it's happening above too. So I think, some product management,   is collapsing into the world of software engineer or engineers are sort of laddering up into product management. And I think part of that is the nature of these deeply technical products   that we're building. So I think many engineers make awesome product managers.  

I mean, perhaps they have to step away and and, you know, be willing not to write as much code anymore.   But because they have an intrinsic understanding of the way these systems are built, I think engineers are just sort of like reaching and and,   absorbing a lot of these other roles. And so some of the best product managers that, you know, I think we've seen, have been ex software engineers. So I just think that there's   a real emerging,   this is just a larger perception that I have of the world, into the software engineering related disciplines. And I think it's actually not a far leap, to sort   disciplines. And I think it's actually not a far leap, to sort of see how, you know, an engineer, a product manager who's informed with an engineering discipline,   is super effective in that role. So I just think this is a broader trend that we're seeing overall, if that makes sense. Yeah. I definitely agree that there has been a much  

[00:15:05] Unknown:

broader definition of what the responsibilities are for any given engineer because of the fact that there are a lot more abstractions for us to build off of. And so it   empowers   engineers to be able to actually have a much greater impact with a similar amount of actual effort   because of the fact that they don't have to worry about everything from the silicon up to the presentation layer because there are so many different   useful prebuilt capabilities that they can take advantage of and think more in terms of the systems rather than the individual bits and bytes. Yeah. Exactly.   And in terms of your overall   efforts for community building and community management,   There are a number of different   sort of focal points for communities to grow around that happen because of different shared interests or shared history.  

So there are programming language communities. There are communities focused on disciplines such as, you know, front end development   or   product management or business. And I'm wondering   what your experience has been in terms of how to   orient a community focus along the axis of data given that it can, in some ways, be fairly nebulous as to   what are the   common principles in data because there are so many different ways to come at it and so many different formats that data can take.  

[00:16:31] Unknown:

Yeah. I think the core, you know, 1 of the core values for us, and I don't know if this is necessarily specific to data or not, but it's just openness. And,   I think especially,   you know, we're we see ourselves as much, much more than just a conference series. And we use the word community,   in our team and at our events,   and just to describe what we're doing,   dozens and dozens of times a week. And so,   yeah, I think the community bond and the mentality   for our brand is super high.   I think that, you know, there's also an open source sort of commitment.   I think that's a mentality. I think that's   a coding,   discipline   and style.  

And I think that, you know, sharing knowledge is just super important in any of these technical fields. And so   engineers are super thirsty for knowledge, and we see ourselves,   as being a connecting point where   engineers and data scientists can come and share knowledge with each other.   I think especially maybe that's a little bit, accelerated   in the case of data science or AI research,   because these things are changing so fast. And so there is a little bit of an accelerator   in terms   of the way that we're able to see our community grow and the interest in the space,   because so much of this technical stuff is changing   quickly.  

And, you know, engineers   need a trusted source to come to where they can find and and get surfaced the best and most interesting research and most interesting open source tools.   So we've capitalized on that and   and we try and be and on 1 hand, we're sort of a media company. You know, on the other hand, we're an events business. On the other hand, we're a community.   But we're sort of putting all these 3 things together,   in a way that we think benefits careers for engineers, and it enables them to level up in their careers and make them smarter and get better jobs and meet awesome people.   So really, all in all, you know, we see ourselves as building this building this awesome talent community,   around data and AI   worldwide.  

And we think we're in a super unique position to do that and succeed at it.  

[00:18:35] Unknown:

Community building   can   be fairly challenging because of the fact that you have so many different disparate   experiences   and opinions coming together.   And sometimes it can work out great. Sometimes you can have issues that come up just due to natural tensions between people interacting in a similar space. And I'm wondering   what you have been drawing on for   inspiration and reference for how to foster and grow a healthy community.   And any interesting or challenging or unexpected aspects of that overall experience of community management that you've encountered in the process?  

[00:19:14] Unknown:

Yeah. I think it's an awesome question because   any company that embraces community,   to some degree,   embraces perhaps,   somewhat of a wild, wild west. And I think some companies and brands   manage that very heavily top down.   And they want to, and they have the resources to, and they're able to.   Some others, I think, let the community sprawl. And, you know, in our particular case, because we're a tiny startup,   I used to say that we're   3 people and 2 PayPal accounts, running events all over the world, you know, even though we're just a touch bigger than that now, not much. But we have 18 meetups all over the world and forming conferences from San Francisco to Singapore.   So I think the only way that we've been able to do that,   and just to be clear, like, we're a for profit business,   but I think that's 1 of the other things that makes us super unique is that, yes, we're for profit,   but at the same time, we're embracing a highly principled notion of community, and we use lots and lots of volunteers   in order to help, you know, further that message worldwide because we can't afford to hire community managers in every single city that we wanna operate in.  

So so that that's 1 thing. And I guess for us,   we've just had to embrace kind of the the wild nature of what it means to scale a community   worldwide and deal with the ups and downs and challenges that come with that. And of course, there's some brand risk,   and there's, you know, other sorts of frustrations,   sometimes working with volunteers. But I guess my inspiration, you know, specifically in this was really through through 500 Startups. And I went on Geeks on a plane back in 2012,   I believe. And when I saw the way that 500 Startups, which is a startup accelerator in San Francisco,   was building community   all around the world, basically 1 plane at a time. And I saw how kind of wild and crazy   that community   was. I sort of   learned, like, the opportunity and the challenge   of of building community that way. And I think the main thing, you know, if you can embrace the chaos,   and if your business situation forces you to embrace the chaos in order to scale,   I think the main way that you keep that in line is   you just have a few   really big core values   that you talk about and you emphasize a lot. Because basically,   the community has to sort of manage itself against those values.  

And, you know, this this isn't like a detailed, like, heavy top down process because you just can't in that scenario.   So I think the most important thing is that the community understands the ethos of what you stand for. And that's why with Data Council,   you know, there's a couple things. I already mentioned open source that's super important to us. And we're always looking for new ways to lift up open source   and to   encourage engineers to submit their open source projects for us to promote them. We prioritize open source talks at our conference.   You know, that's just 1 1 thing. I think the other thing for Data Council is that we've committed to not be an over sponsored brand.  

This can make it hard economically for us to be able to grow and build the hire the team that we want to sometimes, but we're very careful about the way we integrate   partners and sponsors into our events. And,   we don't want to be, you know, what we see as some of our competitors being sort of oversaturated and swarming with salespeople.   So there's a   few, like, high thing. And I guess the other thing that that's super important for us is we're just deeply,   deeply committed to deeply technical content.   We screen all of our talks and,   we're uncompromising   in the in the speakers that we put on stage.  

And I think all of these things resonate with engineers. Like, I'm, I'm an engineer, and so I know how engineers think. And I think these 3 things   have differentiated us from a lot of the other,   conferences and and and events out there.   We realized that there was space for this differentiation. And I think all these things resonate with engineers. And now it makes engineers and data scientists around the world wanna raise their hands and help us launch meetups. We were in Singapore,   last month. We launched our 1st data data council conference there, which was amazing. And the whole community came between Australia and India and the whole region, Southeast Asia. They were there. And we left with 3 or 4 new meetups in different cities because people came to the conference, saw what we stood for, saw that they were sitting next to awesome people and awesome engineers, and and it wasn't just a bunch of suits at a data conference.   And they they wanted to go home and take a slice of data council back to their city. And so we support them in creating meetups. And we connect them to our community, and we help them find speakers. And it's just been amazing to really see that thrive. And I think, like I said, the main the main thing is just knowing   the the core ethos of what you stand for. And even in the crazy times, just being consistent about the way you communicate that to the community, and then letting the community run with it and see what happens. And sometimes it's it's messy and and sometimes it's awesome. And but, you know, it's a it's an awesome experiment. And I just think it's incredible that a small company like us can have global reach. And it's only because of the awesome volunteers, community organizers, meetup organizers,   track hosts for our conference that we've been able to suck into this into this orbit. And we just wanna make the world a better place for them. And they've been super, super helpful and and kind in in supporting us, and we couldn't have done it without them. So it's been an awesome experiment. And, you know, we're we're continuing to push forward with that model.  



[00:24:34] Unknown:

With so many different participants   and the global nature of the community that you're engaging with. There's a lot of opportunity for different surprises to happen, and I'm wondering what have been some of the   most interesting or exciting,   to paraphrase Bob Ross, happy accidents that you have encountered in your overall engagement with the community?  

[00:24:57] Unknown:

I guess   this wasn't totally unsurprising,   but   I just love to sort of surround myself with with geeks. You know, geeks have always been my people. And   even when I stopped writing code actively,   I just gravitated towards software engineers. And, obviously, which is why I'm sort of, you know, I do what I do, and it's what makes me tick.   I guess 1 of the interesting things through running a conference like this is you get to meet such awesome startups.   And there's so many incredible startups being started outside of the Valley. You know, I lived in New York City for many years, and I lived in San Francisco for many years, and now I spend most of my time in Wyoming. So I'm relatively off the map   in 1 way of thinking, but in the other way, you know, as the center of this conference, we just meet so many awesome engineers and startups all over the globe. And I'm really happy to see such awesome ideas start to spring up from other startup ecosystems. So, you know, I don't believe that all the engineering talent should be focused in Silicon Valley, even though it's easy to go there,   learn a ton,   really benefit from that, benefit from the community, benefit from the the the big companies with scale. But, ultimately, I think, you know, not everyone is gonna live in the Bay Area. I I hope they don't because it's already getting a little messy there. But I just wanna see, you know, all of these things sort of democratized and distributed,   both in terms of software engineering and then the engineers that start these awesome startups. And so, you know, the the ease with which I'm able to to meet and see new startups around the globe through the data the data council community, I think, has been a real bright spot in that. And, I don't know if it was necessarily a surprise, but maybe it's been a surprise to me at how quickly it's happened.  



[00:26:35] Unknown:

So 1 of the other components   to managing communities is the idea of sustainability   and   the overall approach to governance and management. And I'm wondering both for the   larger community aspect as well as for the conferences and meetup events, how you approach sustainability   to ensure that there is some longevity   and continuity   in the overall growth   and interaction with the community?  

[00:27:03] Unknown:

Yeah. I think I think the main thing, you know, this gets back to another core tenant of sort of the psychographic of a software engineer.   Software   engineers need to know how things work. And that's sort of at the core of our mentality in in building things.   We wanna know how things work. If we didn't build it ourselves,   we'd prefer to, like, rip off the covers and understand how it works.   And, you know, to be honest, part of the way that, for instance, we select talks at our conference,   you know, I think this applies to, and we're we're learning to get better about, I mean, I think as a as a value, we believe in openness and transparency,   in our company. I think, externally facing, we're getting better about how we actually enable that with the community. But for instance,   for our next data council conference that's coming up in New York in November,   we've published   all of our track summaries   on GitHub, and we've opened that up to the community,   where they can contribute   ideas,   questions,   maybe even speakers,   themes, subthemes, etc. And   I think just the nature that, you know, we have the culture to start   to   community that's just sort of inherent, I think, in a well run community is the amount of diversity you get.  

And obviously, you know, we're all aware of that, that software engineering as a discipline   is just suffering from a shortage of diversity in certain areas. And   I think, as we commit to that   locally, regionally, globally,   there's so many types of different diversity   that we get through the event.   So I think both of these things are, you know, are super meaningful in, like, keeping the momentum of that community moving forward because   we wanna continue to grow and we wanna continue to grow,   by welcoming folks that maybe necessary didn't previously identify,   with the data engineering space, you know, into the community so that they can see what it's like and evaluate if, if they wanna take a run of that in their career. So I think, all these things,   transparency, openness, diversity,   these are all hallmarks of a great community. And, and these are the, the, the, the engines that keep the community going and moving forward.  

Sometimes in spite of the resources or lack of resources,   you know, that a company like Data Council itself   can muster at any 1 time.  

[00:29:22] Unknown:

In terms of the conference itself, the tagline that you focused on, and we've talked a little bit about this already, is that they are no fluff paraphrasing your specific wording. And as a way of juxtaposing them against some of the larger events that are a little bit more business oriented,   not calling out any specific names.   And I'm wondering what your guidelines are for fulfilling that promise and why you think it is an important distinction,   and conversely,   what some of the benefits are for those larger organizations,   and   how the 2 scales of event play off each other to help foster the overall community?  



[00:30:03] Unknown:

Yeah. Well, 1 1 thing here is I think comes to the mentality of the engineer, and then the other side of it is the mentality of the sponsor and the partner.   And, you know, Hey, I think engineers are just noble. And,   like I said, engineers want transparency. They want to know how things work. They don't want to be oversold to, you know, they want to try a product for themselves.   There's just all of these sort of things baked into the engineering mindset. And first and foremost, we want to be known as the conference and the community that respects that. Like that's the main thing because engineers, like without engineers in our community,   loving it, getting to know each other, we're not careful about the the the opportunities and the context that we've created for them. They're just gonna run-in the other direction. And so, like, 1st and foremost, like, those are the hallmarks of of what we're building from the engineering side. Then on the partnership side, I think companies are not great at understanding how engineers think.  

Recruiters are not great at talking to engineers.   Marketers are not great at talking to engineers. I mean, yes, engineers need jobs, and yes, engineers need new products and tools. But to find companies   that actually know how to respect the mental hurdles that engineers have to get through,   in order to like get interested in your product or get interested   job,   you know, that's a super significant thing. And,   through my years of working in this space,   I've done a lot of coaching and consulting with CTOs,   specifically surrounding these 2 things, recruiting and,   and marketing to engineers. And   I think that awesome companies   who respect the the central place that engineers have and will continue to have   in in the innovation economy that's coming,   realize that they have to change their tune in the way they approach these engineers.  

So I, you know, our conference   platform   is a mechanism that we can use   to gently sort of steer and even teach,   some partners   how to interact with engineers in a way that doesn't scare them away. And so just broadly speaking, like I mentioned, we're we're just super careful about how we integrate partners with our event. And we're always, as a team, trying to come up with with better ways to message this and and better ways to educate and and sort of welcome sponsors, you know, into the special   the special network that we've built. But it's challenging. You know? Like, not not all marketers think alike, and not all marketers know how to talk to engineers. But we're committed to creating a new opportunity   for them to engage with awesome data scientists and data engineers   a way that's valuable for both of them. And that's a really fun,   big challenge. And, you know, we're not as worried about how much as it scales right now, as much as we were the quality,   enhancing the quality of those interactions. And so that's what we're committed to as a brand. And, you know, it's not always easy, but we've we've learned a lot and we have a lot to learn. And we always sort of touch touch base with the community after the events and sort of ask the community what they thought and how they they interact with the partners. And did they find a new job and how did that happen? And so we're always trying to pour gasoline on what works in that respect and continue to innovate and move forward in that way. In addition to your work on building these events and growing the meetups and overall network  

[00:33:11] Unknown:

opportunities for people in the engineering space. You have also become   involved as an investor, and you've mentioned that you focus on data oriented businesses.   And I'm curious how you ended up getting involved in that overall endeavor and how that fits into your your work with Data Council and what your overall personal mission is. Oh, yeah. Well, that's that's definitely 1 of my side projects.  

[00:33:34] Unknown:

As I mentioned, I wanna help a 1000 engineers start companies. And,   you know, this is just part of what makes me tick. Just helping software engineers through the conference,   through advising their startups, you know, through investing, through helping them figure out go to market. I guess a lot of this this energy for me came, you know, from having started 4 companies myself   and as an engineer who didn't go to B school.   But instead opted to start, you know, my first company in New York City in 2003.   You know, there weren't a lot of people that had started 3 companies in New York by the time,   the early, you know, thoughts, sort of,   or or the late came around. So, yeah, I guess I've I've learned a lot of things the hard way. And I think a lot of engineers are kind of self taught and they also learn things. They tend to learn things the hard way. So I guess   a lot of my passion there   is again, sort of meeting engineers where they're at, how they learn. And, you know, to them, like, I'm kind of a business guy now. I have experience with starting companies,   building products,   fundraising, marketing, building sales teams. And, you know, most of those things,   are not necessarily   on top of mind for many software engineers that wanna start a company. They have a great idea. Idea. They're totally capable of engineering it and building a product, but they need help in all the other, you know, software business y, sales y stuff,   as well as fundraising. So I guess I've just figured, I've discovered the sort of special place I have in the world where I'm able to help them, coach them,   through a lot of those business challenges. I I could never build the infrastructure that they're building or figure out the the algorithms or the models that they're building, but I can help them figure out, how to pitch it to investors, how to pitch it to customers,   how to go to market,   how to hire a team that scales.  

And so I've discovered that I just had, you know, through my ups and downs as an entrepreneur,   I've developed a large set of early stage hustle,   experience. And I'm just super happy to to pass that on to other engineers who are also passionate about starting companies. So that's just something I found myself doing anyway, you know, as a mentor for 500 startups or as a mentor for other companies.   And 1 thing led to another, and soon I started to do angel investing. And now I have an AngelList syndicate,   which is actually quite interesting because it's backed by a bunch   of CTOs and heads of data science and engineers from our community who all co invest with me.  

And as I'm able to help, companies bring their products to market, startups,   come to market.   Oftentimes there'll be an investment opportunity there. And so I'll bring in another network of technical people,   who add value to that company even more.   So I'm just sort of the, you know, a connector in this community. And the community is doing all kinds of awesome stuff, you know, inside and even, sometimes outside of data console, which is just a testament to the power of community overall. And, you know, I just happen to be, super grateful that I'm along for the ride and I get to I get engineers who who come to me and trust me for help, and I'm able to to connect these dots and, and help them succeed as well. In terms of the  

[00:36:33] Unknown:

ways that businesses are structured,   I'm wondering   what it is about   companies that are founded by engineers and led by engineers   that makes them stand out and why you think that it's important that engineers start their own companies, and how that compares to businesses that are founded and run by people who are coming more from the business side, and just your overall experience in terms of the types of focus and levels of of success that those 2 different,  

[00:37:01] Unknown:

sort of founding patterns end up playing out? Yeah. Well, you I mean, you can tell based on,   what I've been saying that I'm just super bullish on the software engineer. And,   you know, does that mean that the software engineer as a persona or a mentality or a skill set,   you know, is is inherently awesome and has no weaknesses and no problems? Like, hell hell, no. Of course not.   And I think the some of the the the challenges of being a software engineer and how your mentality fits into the rest of the business are well documented. So   I think, all of us as engineers need to grow and diversify   and increase the breadth of our skills. And so that has to happen. But on the other hand, if we believe that innovation is going to continue to sweep   the world and,   highly technical   solutions,   perhaps to sometimes non technical problems, perhaps sometimes the technical problems,   are going to continue to emerge.  

I feel like people who have the understanding   of of   the the the technical implications   and the realities and the architectures and the science of those solutions,   just have an interesting edge. So I think there's a lot of hope in teaching software engineers how to be great CEOs. And I think that's that's increasingly happening. I mean, look at,   Jeff Lawson from Twilio or the guys from Stripe.   Even if Uber was started by an engineer. Right? There was the the the quiet engineer at Uber at Uber, Garrett,   sort of quiet in terms of Travis,   you know, who who was a cofounder of that company. So I think we're seeing engineers,   not just start   the most amazing companies that are, that are changing the world, but they're increasingly in positions   of becoming CEOs in those companies. You know, I guess you might even take that 1 step further. And I'm kind of trying to be   an engineer who's also been an operator and a founder. But now I'm I'm stepping up to becoming a a a VC and,   and being an investor. So I think there's the engineer VC, which is really interesting as well, but I think that's a slightly different, conversation. But but suffice it to say that engineers,   are bringing a valid mentality,   into all of these disciplines. And, yes, of course, an engineer has to be taught to think like a CEO, and and has to learn how to be strategic, and has to learn sales and marketing skills. But,   I think it's just an awesome awesome challenge to be able to layer those skills on top of engineering discipline that they already have. And,   I'm not saying this is the only way to start a company or that business people,   you know, can't find awesome engineers to partner with them. I mean, honestly, I think an engineer often needs a a business cofounder,   to help get things going. But I I I'm I'm coming at it from the engineering side and then figuring out, like, all the other support that the engineer needs to make a successful company. That's just because I've chosen that particular way, but other people will be coming at it from the business side, and and I'm sure that will be fine for them as well. In terms of the challenges that are often faced for an engineering founder in growing a business and making it viable, what are some of the common points of conflict  

[00:40:00] Unknown:

or,   misunderstandings   or challenges that they encounter? And what are some of the ways that you typically work to help them in establishing the business and setting out a path to success? Well, I think the the the biggest thing, you know, that I see is,  

[00:40:15] Unknown:

many engineers are just afraid to sell. And unfortunately,   you know, you can't have a business,   if you can't have some sales. And so somehow,   engineers have to get over that hurdle. And   that can be a long process. It's been a long process for me.   And I still undersell   what we do at Data Council, to be honest, in some ways. And I have people around me to help me do that. And we wanna do that, again, in a way that's in line with the community. But,   I'm constantly trying to figure out,   how to be   essentially a better salesperson. But for me, that means that still retaining sticking to the core of my engineering values, which   is honesty,   transparency,   enthusiasm,   you know, value, and really understanding how to articulate the value   of what you're bringing in a way that's that's unabashedly   honest and transparent.  

So I think that's a that's a really big thing for a pure engineer founder is it can be difficult to go out there and figure out how to get your first customers.   You know, how to start to sell yourself personally.   And then the next step is is how do you build a sales culture and a process and a team   that's in line with your values as a company? And that scares, you know, that scares some engineers because it's just terrifying to think about building a sales org when you can barely accept that your product needs to be sold yourself.   But I think that's just, you know, that's sort of ground 0 for starting a company. And so, you know, I I try and be as gentle as possible in in sort of guiding engineers through that process. But I guess that's the 1 of the core hiccups that I think engineers have to figure out how to get over by   bringing in other people they trust or getting advice or, you know, you have to approach it sometimes with kid gloves. But teaching engineers how to sell in a way that's true to their values,   I think is just a really big, big elephant in this room that, you know, I constantly run into and and and try to help, engineer founders with.  



[00:42:09] Unknown:

In terms of the businesses that you evaluate for investing in, what is your   characteristics   of a business that you look for when deciding whether or not to invest? And then as a corollary to all of that,   why is it that you're focusing primarily on businesses that are focused mainly in the data space   and the types of services that you're looking for that you think will be successful and useful? Yeah. Well, I guess last question first.  

[00:42:40] Unknown:

I think, you know, it's important to have focus   as an investor.   And   not everybody can do everything awesome.   I think it's also a strategy to building a new brand   and a niche fund in the face of   the Sequoias   and the Kliners of the world.   I I think we might have, like we might be past the days when   a new fund can easily build a brand that's that's that expansive.   So I think,   this is just kind of, you know, typical marketing strategy.   I think if you start to focus on a niche and do 1 niche really well,   I think that produces   network effects, smaller network effects inside that niche that then grow and grow and grow and develop.  

So, I mean, I've chosen to focus, in my professional life on data. Data, data science, data engineering, data analytics. That's data   council. Partly, that's because I just believe in the upside of that market.   So I think that's just a natural analog to a lot of my investing activity. Because I'm knowledgeable about the space, because I have a huge network in the space,   because I'm I'm looking and interested in talking to these companies all the time. It's just a natural fit. That's not to say that I don't I mean, I'm also passionate about broader developer tools. And as you mentioned earlier, I'm an investor in Clubhouse.   I'm interested in security companies. So I think, you know, for me, there are some analogs to to just, like, deeply technical companies, you know, led by technical founders that that, also fit my thesis. But, you know, still, it's a fairly niche, narrow thesis. Like, most of the stuff I do,   on the investing side is b to b. I meet the companies through my network and and through data council.  

I think they're solving, you know, meaningful problems in the b to b space. Another criteria   often is they may be supported by some,   previous open source success or the company might be built on some current open source project   that I feel gives them an unfair advantage when it comes to distribution,   and getting a community excited about the product.   So these are a few of the things that that I look for, in terms of of the investing thesis.  

[00:44:40] Unknown:

In terms of the overall industry   and some of the trends that you're observing in your interaction   with engineers   and with   vetting the different conference presentations   and meetup topics, what are some of the trends that you're most excited by? And what are some of the ones that you are either concerned by or   some potential issues that you see coming down the road in terms of the overall direction that we are heading as far as challenges that might be lurking around the corner? Well, I think, you know, 1 big thing there is just, data science and ethics.  

[00:45:16] Unknown:

AI fairness,   bias in AI and and deep learning models,   ethics when it comes to, you know, targeting customers, what data you keep on people. Like, I think all these things are just super   interesting   is business issues. They're policy issues or business issues.   At 1 level, they're also technical issues.   So there's technical implementation stuff that's required. But I just think, raising that discussion   is important. And so that's 1 area that we're focusing on and data council in the next, series of events that we run later this year and next year is   elevating that content,   in front of our community so that it can be a matter of discussion because I think those are important topics. They're not always seen as the most technical.  

But I think they're super important in terms of us trying to help the community   steer and figure out where the ship is going in the future. So I think that's super interesting. And then I guess on the technical side, I think the data ops world is starting to to mature.   I think that there's a lot of interesting opportunities there in the same way that the DevOps revolution,   you know, revolutionized the way that software was   built, tested, deployed, monitored.   And companies like Chef and New Relic, you know, came came out in perhaps the mid 2000s.   I think we're at a similar inflection point with data ops, if you will. And   there's more repeatable pipelining process.  

There's, back testing   and, and auditing,   capabilities that are required for data pipelines that aren't always there. There's monitoring infrastructure that's being built,   and some some cool companies that I've seen recently.   So I think data ops and basically just elevating   data engineering to the level that software engineering has been at for a while,   is definitely something that seems to be catching fire. And and we also, you know, try and support through the conference  

[00:47:07] Unknown:

as well. Looking forward, what are some of the goals and plans that you have for the future of the data council   business   and the overall community and events?  

[00:47:17] Unknown:

Well, I think, as I mentioned, our our biggest goal,   is to build   the data and AI talent community worldwide.   And I think that there's,   we're we're building a network business. So,   I guess it it's kind of takes me back to when I started   Hacklabs, which was the digital social network. And I thought we were building a digital product.   And as I already mentioned, 1 thing led to another, and now we have data council instead. Well,   data council is, you know, butts in seats at events and community, and it's IRL,   engineers getting together, but it's still a network. It's not a super fast digital thermal   network,   but it's a network, and it's a super meaningful network. So it's kind of interesting that after all the ups and downs, like, I still see ourselves as in the network building business. And I think the cool thing about building a network is once you build a network,   there's lots of different value that you can add on it. So I think in terms of there's there's really big ideas here. There's formalizing   education and education offerings.  

There's consulting services that can be built on top of this network,   where companies can help each other,   or recruiting sort of fits in the same vein.   I think there's there's other things as well. There's,   a fund, which I mentioned is a side project that I have,   to help engineers in this community,   start companies.   So I think there's there's all kinds of interesting   things that you can build on top of a network   once you've gotten there. And   for now, you know, our events are essentially a breakeven business that just give us an excuse and the ability to grow this network   all around the world globally.  

But I think, you know, there's a much bigger, like, phase 2 or phase 3 of this company where we can build really awesome stuff based on this engineering network,   a network and a brand that engineers trust that we've laid down, in the, in the early part of the building phase. So I'm really excited to see that and, and, and develop that strategy and, and mission going forward.  

[00:49:14] Unknown:

Are there any other aspects of your work on data council or your investment   or your overall   efforts in the data space that we didn't discuss yet that you'd like to cover before we close out the show?  

[00:49:27] Unknown:

No. I think I think we covered a lot of stuff.   I hope it was interesting for you and and your audience. And I encourage folks to reach out to me and and get in touch. If there's engineers out there that wanna start companies,   if there's engineers that wanna participate in our community worldwide,   we're always looking for awesome people to help us find and screen talks.   We're we're interested in awesome speakers as well.   I'm always interested in talking to, deep learning and AI researchers who are out there who might have ideas that they wanna bring to market. But, yeah, you can reach me at pete@datacouncil.ai,   and I'm happy to plug you into our community.  

And, yeah, if I can be helpful to anyone out there, I would just, really encourage them to to reach out.  

[00:50:09] Unknown:

And for anyone who does want to follow-up with you or keep in touch or follow along with the work that you're doing, I'll have your contact information in the show notes. And as a final question, I I'd just like to get your perspective on what you see as being the biggest gap in the tooling or technology that's available for data management today.  

[00:50:26] Unknown:

Yeah. I think, as I mentioned, I think maybe it comes down to this this data ops thing. Right? There's there's really interesting open source projects coming out, like Great Expectations   is 1.   Interesting   companies coming out like elemental,   which is built around DAGSTER, which is an open source project.   So I think that this is a really interesting   niche sort of tooling area,   specifically in the data engineering world that I think we should all be watching.   And then I guess the other sort of category of of tooling I'm seeing, it's sort of related. It's also in the monitoring space. It's   watching the data in your data warehouse,   to see if there's anomalies that sort of pop up because we're all,   we're pulling together data from so many different hundreds of sources now,   that I think it's a little bit tricky   to watch for data quality and integrity. And so I think there's a new suite of tools that are popping up in that data monitoring,   space, which are very interesting. So those are a couple of areas that I'm interest that I'm interested in and looking at, especially when it comes to data engineering applications.  



[00:51:27] Unknown:

Well, thank you very much for taking the time today to share your experiences definitely   an   definitely an area that I find valuable, and I've been keeping an eye on your conferences. There's been a lot of great talks that come out of it. So I appreciate all of your work on that front, and I hope you enjoy the rest of your day. Yeah. Thanks, Tobias. We'll see you at the data console conference sometime soon.   Listening. Don't forget to check out our other show, podcast.init@pythonpodcast.com   to learn about the Python language, its community, and the innovative ways it is being used.   And visit the site at dataengineeringpodcast.com   to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show, then tell us about it. Email hosts at data engineering podcast.com   with your story. And to help other people find the show, please leave review on Itunes and tell your friends and coworkers.