Finding lane lines with Computer Vision

Finding lane lines with Computer Vision

I recently joined the Udacity Self Driving Car Nanodegree. The nanodegree is amazing and I’m really enjoying taking it. The nanodegree is comprised of several projects and the first project (and the simplest one) is to build a pipeline to identify lanes on a road. Here I’m gonna present my solution to this project. The code is written in Python and I’m using the OpenCV library. The full code is available in a python notebook in my Github.

Identifying lane lines is a common and simple task that self driving cars need to perform. There are a bunch of techniques out there. Since this is the first project, we’re suppose to build a more simpler approach. Our goal here is to identify straight lines in order to “fit” left and right lane markings. This approach works well when a car is driving straight but it starts to perform badly on curves (and you’ll see an example of this at the end of this article).

Overview of the pipeline

Before digging into each step taken, let me summarize what I’m doing in order to identify the lanes markings.

  1. Convert Images to HSL
  2. Apply a white and yellow color masks to the image
  3. Apply a Gaussian filter
  4. Run Canny Edges detection to identify the edges
  5. Get the image ROI (region of interest) to focus on the road
  6. Run Hough Transform to find lines in the image
  7. Separate lines into left and right lines
  8. Find the best fit line for the left and right line points
  9. Extrapolate the lines in order to get one single line that cover the entire lane line

Pre-processing the image

The first three steps does some pre-processing to the image that will make our life easier in the next steps. First I convert the image color space from RGB to HSL, this gives us slightly better results for color thresholding compared to RGB. HSL is just another model for representing colors. There are several image color models such as RGB, YUV, YCrCb, CMYK, I’m not gonna dive into details of color representation but you can think of them as different ways to represent colors in images. It’s also important to note that there are equations that can convert colors from one representation to another. If you’re interest in reading more check out this Wikipedia article. in OpenCV converting color models is as simple as calling a function:

img = cv2.cvtColor(image,cv2.COLOR_RGB2HLS)

Next, I apply a technique called color thresholding to select pixels whose color intensity values are between the given color range. For example, if we want to select only white regions of the image, we could look for pixels with RGB values of (255,255,255) since it represents white. However, there are various levels of white and if we only look for the “full” white we would be missing tons of pixels in the image. A better approach is to get pixels within a given range like (200,200,200) to (255,255,255). The sequence of images below shows the result of applying two color masks to get yellow and white lane markings. The last image is the two masked combined.

I then use the combined mask to get only the pixels of interest in the original image. The image on the left is the result after applying the Gaussian filter which basically blurs the image. The goal of applying the Gaussian filter is to remove noise and avoid noisy edges to be detected. Compare the images below, you should be able to see that noisy pixels around the lane markings have been smoothed out.

Finding the edges

Now that I have filtered out most of the pixels of the image that I’m not interest in, I run the canny edge detection algorithm. Basically it uses the gradient of the image with a low and high threshold parameters to find edges. If you’re interest in learning more about Canny Edge check out this link. I used 50 and 150 for the low and high threshold respectively.

The canny edge algorithm returns a binary image with white pixels belonging to the edges the algorithm detected. There are still some unwanted edges found in the image, but since the camera is always in a fixed position we can simply define a region of interest and discard everything outside this region.

Cool! with a region of interest I got only pixels belonging to the lane markings! The next steps is to find actual line points from this image.

Hough Transform and extrapolating lines

This is probably the most complicated step. Here I first apply the Hough Transform Algorithm. Basically, it receives the output of any edge detection algorithms such as canny edges and tries to find lines. The algorithm is a bit complex but the way it works is beautiful! Check out this article if you’re interest in learning how to implement it. For applying the Hough Transform I decided to use following parameters:

  • Rho: 1
  • Theta: PI/180
  • Threshold: 30
  • Min Line Length: 10
  • Max Line Length: 150

Initially I was using lower values for the Max Line Length, but increasing it to a higher value improved the line detection as it was able to “connect” the dashed white lane markings. Bellow is the output of the raw lines detected plotted onto the original image.

As you can see it detects multiples lines for each lane markings. So we need a way to average these lines. The idea is to find two lines, one for the left lane and another for the right lane. So the first step is to separate the lines found. The way we can do that is by computing the slope of the line:

  • If slope is positive (as y increases, x increases as well), the line belongs to the right lane.
  • If the slope is negative (as y increases, x decreases), the line belongs to the left lane.

Once the lines are properly grouped, I use line the points to find a best fit line using the least square method. In python it’s possible to easily calculate that by using np.polyfit(). This function will return the coeficientes of the line equation (y = ax + b). With this equation I can easily plot a line from the bottom the of region of interest to the top of it. The image below shows the final output of the pipeline, previous steps are also shown for better understanding.

Tests on videos

Video 1

Video 2

Video 3

This video is more challenging because it contain curves. Since we’re only using straight lines the solution does not perform very well, but it is still able to detect the lane lines.

Areas to improve

  • Straight Lines aren’t able to detect lane lines properly in curves. One potential solution is to use quadratic functions to fit the lane markings.
  • The current pipeline always assume that lane markings are either yellow or white. By parameter tuning it should be possible to completely get rid of color masks and use only edge detection.
  • Use some sort of memory to average the current line with previous lines in order to make the detected lines more stable.


Como é trabalhar remotamente?

Como é trabalhar remotamente?

Muitas pessoas me perguntam como é trabalhar remotamente, e já algum tempo que eu procuro reservar um tempo para escrever este post. Pois bem, finalmente arranjei um tempo (6 horas de espera em um aeroporto). Tanto o trabalho remoto como a oportunidade de trabalhar para empresas do exterior, seja remoto ou não, tem atraído o interesse de muita gente, principalmente na área de computação. Neste post vou tentar explicar um pouco como funciona esse modelo de trabalho, baseado principalmente na minha experiência e também em relatos de colegas que trabalham para outras empresas.

Para quem quer ler sobre como comecei a trabalhar remotamente, recomendo a leitura do artigo Serviço Público, Home-Office e a 10up. Como larguei o serviço público para trabalhar para os gringos.

Continue reading “Como é trabalhar remotamente?”

WordPress and GraphQL

WordPress and GraphQL

GraphQL is a emerging technology developed internally by Facebook and open-sourced right after. It is aimed to be a replacement of the REST API and more and more developers are buying the idea. GraphQL poses a declarative way of fetching and update data and thus simplifying the process of doing so on the front-end. Dealing with the server side of a GraphQL stack might be a bit more complex than building REST API endpoints, but the benefit is worth the challenge and fortunately there are several libraries out there that can help you build GraphQL servers.

Continue reading “WordPress and GraphQL”

FullStack London 2017: A review

FullStack London 2017: A review

This year I had the privilege to travel to London and attend the FullStack London Conference. FullStack London is a conference on Javascript, NodeJS, React, IoT and much more. I was sponsored by 10up (the company I work for). 10up has a really great initiative that sets aside a budget for all of its employee for self-professional development. Some uses for taking courses, other uses for attending conferences.

The main reason that led me to choose this conference was due to my personal goals of improving my Javascript related skills. Javascript is being used more and more, not only in the browser but everywhere (desktop apps, native mobile apps, games, VR and etc). It’s hard for a developer to keep up with the fast pace of javascript and a conference like this one is great to get a sense and a learn about the latest technologies.

Without further ado, let’s get to the review.

Continue reading “FullStack London 2017: A review”

Serviço Público, Home-Office e a 10up. Como larguei o serviço público para trabalhar para os gringos.

Serviço Público, Home-Office e a 10up. Como larguei o serviço público para trabalhar para os gringos.

O objetivo inicial deste post era falar um pouco sobre como é trabalhar no formato home-office na 10up (uma agência web americana), porém vi que tinha muita coisa que poderia ser compartilhada sobre todo o processo de entrevistas e transição, que inclui um pedido de demissão de um cargo público “estável” em uma Universidade federal. Por este motivo o post sobre como é trabalhar na 10up e no estilo home-office ficará para outro dia.

Continue reading “Serviço Público, Home-Office e a 10up. Como larguei o serviço público para trabalhar para os gringos.”

Ano Novo, Blog Novo

Ano Novo, Blog Novo

Há muito tempo, nos anos em que eu descobri que computação seria a área que eu seguiria, iniciei um blog focado em Linux. Meses mais tarde conheci um outro blogueiro também da área e decidimos criar o blog I/O Tecnologia. O projeto durou por alguns bons anos e teve até bons resultados, chegando a ter parceria com Olhar Digital, prêmios e até posts patrocinados. Tínhamos uma média de acessos razoável e foi lá que eu lançei meus primeiros cursos em vídeo aulas: Um mini-curso de Shell Script e o Curso de C, que juntos já somam mais de 100 mil visualizações.

No auge tínhamos 3 autores fixos e alguns convidados com uma média 1 post por dia. Infelizmente o projeto foi ficando de lado e hoje a única coisa que restou dele foi o domínio que é usado para endereços de email dos seus respectivos criadores.

A criação desse blog foi estipulado como meta para o ano de 2017. Dentre os meus objetivos principais estão compartilhar conhecimento e minhas experiências como Web Engineer (ou Desenvolvedor Web como chamam aqui no Brasil…) e também um pouco do que já estudei e ainda estudo na área acadêmica a nível de Graduação e Mestrado em Ciência da Computação.

Em breve novos posts.

Feliz 2017 a todos.