Skip to main content

Posts

Nutch crawler and integration with Solr

Before moving ahead with this article, I assume you have Solr installed and running. If you would like to install Solr on windows, mac or via docker, please read Setup a Solr instance . There are several ways to install nutch which you can read from Nutch tutorial , however I have written this article for those who would like to install nutch using docker. I tried finding help on google but could not find any help for nutch installation using docker and spent good amount of time fixing issues specific to it. Therefore I have written this article to help and save time of other developers. Install nutch using docker- 1. Pull docker image of nutch using below command,      > docker pull apache/nutch 2. Once image is pulled, run the container,      > docker run -t -i -d --name nutchcontainer apache/nutch /bin/bash 3. You should be able to enter in the container and see bash prompt,      > bash-5.1#  Let's setup few important settings now- 1. Goto bin folder,       > bash-5.
Recent posts

Identify security leaks in code

  We will be using Gitleaks software to find security leaks. Here are the steps to identify leaks in your code repo - Install gitleaks on your local machine. Click here to install . Check if it is installed successfully on your local machine.  Website> gitleaks --version If its installed, it should show version of gitleaks. Download sample.config file( see attachment ) and copy on your local machine( this is the sample file with configuration/rules, you may change it as per your requirements ). Now git clone the repo on your local machine and goto your code folder. Once you are in code folder, run gitleaks command. Website> gitleaks --path=./ --config-path=../sample.config --verbose where, --path=./ => path of .git folder --config-path=../sample.config => path of sample.config file You can also download the report of leaks for sharing it with your team - Website> gitleaks -v --pretty --path [RepoPath] --config-path=../sample.config --report= [PathtosaveReport] wher

Interview Questions for experienced professionals: Part 1

This article is for those who are prepaing for any technical job interview. Different organisation have different interview process and here I am sharing the interview of one of the consulting firms. Interview was categorised into multiple sections to understand T shaped skills. T shaped skills accesses a candidate on core expertise(depth of understanding) and other technical skills(breadth of concepts).  I have catagorised the interview into sections which includes, Algorithms, SQL Server, Innovative Ideas, Technical Knowledge and so on. You may check your knowledge as well. Try it once. Algorithms Q: How to swap the values of two integers with using additional memory?  Ans:  public static void SwapInt () { int a = 25 ; int b = 35 ; a = a + b ; b = a - b ; a = a - b ; Console . WriteLine ( "a=> " + a + " b=> " + b ); } Q: How to find duplicate i

Dockerize a dotnet core application with SQL connectivity

Before reading this article, I am assuming that you know Docker, Dotnet core and have a dotnet core application which is trying to connect to SQL server. Read how to build aspnet core app, docker and run the docker container. If docker container is running and you are not able to connect to database, this blog should help you fix it.  Prerequisite -  Make sure code is working via running aspnet core locally via visual studio or command line. Port 1433 is opened for connecting to SQL server. Solution If you have Docker file ready, it should somewhat look like below file -  FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build-env WORKDIR /app # Copy csproj and restore as distinct layers COPY /SampleAPI/*.csproj ./ RUN dotnet restore # Copy everything else and build COPY . . WORKDIR /app/SampleAPI RUN dotnet publish -c Production -o publish # Build runtime image FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 WORKDIR /app/SampleAPI COPY --from=build-env /app/SampleAPI . WORK

Know the rasa ecosystem and train your model effectively

Hope you have read the previous blog to create chatbot using Rasa. I would recommend to read it before starting this blog. This blog will help you understand insights of rasa ecosystem and explain how to train your model effectively. Rasa is an open source framework for creating chatbot with natural language undertsanding. There are few important files of rasa project - ✪ domain.yml This file contain information about intent and respective actions . For instance- basis on intent of a user, appropriate action gets triggered and response is sent back to user. '-utter' is plain text without any logic behind it, however we can create custom action which we will discuss further in the blog. intents : - greet - goodbye - affirm actions : - utter_greet - utter_cheer_up - utter_did_that_help responses : utter_greet : - text : Hey! How are you? utter_cheer_up : - text : 'Great, carry on!' utter_did_that_help : - text : Did that help yo

Create chatbot in 20 minutes using RASA

This blog will help you create a working chatbot with in 20 minutes. For creating chatbot we need following libraries to be installed- >> Python3 >> Pip3 >> Rasa Lets start installing all libraries & dependencies which are need for creating chatbot. Note: I have used MAC, therefore sharing commands related to it. You can install it on Windows, Linux or any other operating system using respective commands. 1. Install Python3 > brew install python3 > python --version #make sure you have python3 installed 2. Install Pip3 > curl -O https://bootstrap.pypa.io/get-pip.py > sudo python3 get-pip.py If you get issue related to Frameoworks while installing pip, follow below steps -  > cd /usr/local/lib > mkdir Frameworks > sudo chown -R $(whoami) $(brew --prefix)/* Once installed check pip3 version > pip3 --version After python3 and pip3 is succeffully installed, proceed to next steps. 3. Install Rasa > pip

Git merge from one repo to another repo

This blog is for those who are looking for merging code from one repo to another repo. Why I will merge code from one repo to another? I forked from one git repo( may be some public git repo ) and did some cutomization on existing code. Occasionaly, features are being introduced in main git branch and I would like to get all those features in my own git repo. In this scenario, I would like to merge latest changes in my git repo. Here are few simple steps which will merge code from one git repo to another- 1. Clone the repo1(source git repo) > git clone https://github.com/org/repo1.git   > git pull 2. Clone the repo2(destination git repo) > git clone https://github.com/org/repo2.git > git pull 3. Goto repo2 (destination git folder) and checkout your prefered branch > cd repo2 > git checkout master>  4. Use below command to see the remote branch associated with your destination git repo > git remote -v  origin https://github.com/org/repo2.g