Setting GitHub hooks in Jenkins

While setting up a Freestyle project in Jenkins we may need to set Github webhooks in Jenkins so that Jenkins will get notified each time there is a push and subsequently fetch the code and build it. Basically the procedure is we set up a URL in Github and it will send a post request to this URL when someone pushes some code. There are 2 ways to get this URL in Jenkins

I have assumed that Jenkins is listening to the default port 8080

Getting the Notification URL

Method 1

Jenkins provides a common URL which can be used as the notification URL. By default the URL will be

http://<IP ADDRESS>:8080/github-webhook/

You can override this URL by going to Manage Jenkins -> Configure System and under Github click Advanced and check "Specify another hook URL for GitHub configuration"

when creating your freestyle project you must specify the build trigger as follows

Method 2

This method though a bit complex allows you to get a URL for each project (you can send a post request to this link from anywhere even from postman to start a build). On the left side, panel click on People (If nothing shows up click on All People option)

* Click on your User ID and click on Configure (on the left side)

* Click on Add new token under API Token. Give it a name and click Generate. Copy the generated token to somewhere safe (NOTE you won't be able to retrieve it later)

* Save

when creating your freestyle project you must specify the build trigger as follows (Set your secret Authentication Token)

The Notification URL will be as follows

http://<JENKINS USERNAME>:<API TOKEN>@<IP ADDRESS>:8080/job/<PROJECT NAME>/build?token=<AUTHENTICATION TOKEN>

Setting the URL in GitHub

To set the URL in Github, go to your project

Confirm it is you by entering your password when asked

That's it now Github will send a post request to the mentioned link each time some code is pushed

Creating a Flutter library for native android applications

Flutter side

1) Use the command flutter create -t module hello_world_module to create the project

2) Create your widget

//main.dart
import 'package:flutter/material.dart';
import 'dart:ui';

void main() => runApp(
MaterialApp(
home:chooseWidget(window.defaultRouteName)
)
);

Widget chooseWidget(String route) {
  switch (route) {
    case 'helloRoute':
      return Scaffold(
	  body:Center(
	    child:Container(
		color:Colors.green,
        child: Text('Hello from flutter',style:TextStyle(fontSize:20,color:Colors.black)),
      ) ));

    default:
      return Container();
  }
}

The reason I have used a switch case is because we can pass a parameter from the java side. Hence we can get multiple widgets from the same library.

Note: There are two ways to use this in a java based project

a) To link the source: this is preferred if we make changes to the android app and the flutter library side by side and we have the source code (Read here)

b) To link a built aar file: this is preferred if we have to distribute it as a library without the source code (in the following steps we will look into this)

3) use the command flutter build aar to build the library. The files will be built inside build\host\outputs\repo , this is the path required in your android project. After running the command flutter will display the steps to include the library in a native app, take a copy of it since it will be useful.

Native app side

1) Create an android project in android studio

2) In the app level build.gradle, insert the following outside the android block

repositories {
    maven {
        url '<PATH TO YOUR FLUTTER LIBRARY>/hello_world_module/build/host/outputs/repo'
    }
    maven {
        url 'https://storage.googleapis.com/download.flutter.io'
    }
}

     3) Inside the dependencies block add  

debugImplementation 'com.example.hello_world_module:flutter_debug:1.0'
  profileImplementation 'com.example.hello_world_module:flutter_profile:1.0'
  releaseImplementation 'com.example.hello_world_module:flutter_release:1.0'

4) Inside the android block add

compileOptions {
    sourceCompatibility 1.8
    targetCompatibility 1.8
  }

5) Inside buildTypes add (not sure if this is required)

profile {
     initWith debug
      }

When you save it wait for gradle sync to finish

6) Open activity_main.xml (app\src\main\res\layout), in design mode drag in a fragment. It will bring in the flutter fragment. Switch to code view and give some meaningful id and a tag

android:id="@+id/hellofragment"
android:tag="flutter_fragment"

7) Open mainActivity.java and change the code as

import androidx.appcompat.app.AppCompatActivity;
import android.os.Bundle;
import io.flutter.embedding.android.FlutterFragment;
import androidx.fragment.app.FragmentManager;

public class MainActivity extends AppCompatActivity {

    private FlutterFragment flutterFragment;
    private static final String TAG_FLUTTER_FRAGMENT = "flutter_fragment";
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
       setContentView(R.layout.activity_main);
        FragmentManager fragmentManager = getSupportFragmentManager();
        FlutterFragment flutterFragmentx;
        flutterFragment = FlutterFragment.withNewEngine().initialRoute("helloRoute").build();

        flutterFragmentx = (FlutterFragment) fragmentManager.findFragmentByTag(TAG_FLUTTER_FRAGMENT);

        // Create and attach a FlutterFragment if one does not exist.
        if (flutterFragmentx == null) {
            fragmentManager.beginTransaction().add(R.id.hellofragment, flutterFragment, TAG_FLUTTER_FRAGMENT).commit();
        }
        else {
            fragmentManager.beginTransaction().replace(R.id.hellofragment, flutterFragment, TAG_FLUTTER_FRAGMENT).commit();
        }
    }
}

Take note of the line initialRoute("helloRoute"). This is where we pass the parameter to our library.

References

1. Codemagic
2. Documentation-add Flutter Screen
3. Documentation-add Flutter Fragment
4. Youtube (video is for some older version, codes don’t work but the concept is the same)

Setting up a basic load balancer in Nginx

The aim is to achieve the following, a user will connect to machine1 running on Nginx at port 80. Requests of the form test1/* will be forwarded in such a way that one out of every three requests will be sent to an apache server running on the same machine at port 85. the remaining two requests will be sent to another machine and the response will be sent to the user

Both the machines are running on Lubuntu in Vbox.

Setting up Machine1:

In machine1 Nginx was installed first

Sudo apt install nginx

Followed by Apache2

sudo apt install apache2

Now, apache2 will fail to start since Nginx is already listening on port 80. To change the listening port edit /etc/apache2/ports.conf and change the line Listen 80 to Listen 85

You must also edit /etc/apache2/sites-enabled/000-default.conf and make the following changes

1.  <VirtualHost *: 80> should be changed as <VirtualHost *: 85>
2. change the DocumentRoot from /var/www/html  to /var/www/html_apache

Open folder /var/www/html and CUT and paste index.html to /var/www/html_apache

If you are unable to save changes to files and get a permission error, run your text editor as sudo. If you get errors while copy-pasting index files run the file manager as root. In Lubuntu you can open a terminal and type sudo pcmanfm

Also, create an index.html inside /var/www/html_apache/test1 and include some text so that you know the response is from apache.

Restart apache with the command 

sudo service apache2 restart

Now opening Localhost should point to Nginx welcome page and localhost:85 should show the apache welcome page

Setting up Machine2:

In machine2 you can simply install apache2, no special changes are required. Get the local IP address of machine2 using the command ifconfig (mine was 192.168.1.5). Also, create an index.html inside /var/www/html/test1 and include some text. Although in real scenarios we will be providing the same functionality, for this demo include a different text compared to the one in machine1 so that the request change can be noticed.

Setting up the load balancer:

To set up the load balancer, in machine1 you should edit the file /etc/nginx/sites-available/default (another approach will be to create a new conf file and linking it to sites-enabled, but I will be editing the default file). Find the block that says server before it starts add the block

upstream sampleloadbalancer {
    server 127.0.0.1:85 weight=1;
    server 192.168.1.5 weight=2;
}

here we are creating an upstream block with the name sampleloadbalancer and we define the two locations, the local server at port 85 and the one running in machine2. The weight parameter defines how many requests are sent to each server. YOu can also exclude a server temporarily by specifying it as down. Now finally, inside the server block Find the block that says location / and at the end of that block add a new location as shown below

location /test1/ {
   proxy_pass http://sampleloadbalancer;
}

By default, it follows the round-robin algorithm. However, you can specify a different algorithm as given here.

Setting up a basic reverse proxy using nginx

The aim was to achieve the following, a user will connect to machine1 running on Nginx at port 80. The Nginx will serve the files unless the URL is of the form test1 or test2, if the url is of the form test1 it will forward the request to an apache server running on the same machine at port 85. If it is of form test2/ then the request will be sent to another machine and that response will be served to the user.

Both the machines are running on Lubuntu in Vbox.

Setting up Machine1:

In machine1 Nginx was installed first

Sudo apt install nginx

Followed by Apache2

sudo apt install apache2

Now, apache2 will fail to start since Nginx is already listening on port 80. To change the listening port edit /etc/apache2/ports.conf and change the line Listen 80 to Listen 85

You must also edit /etc/apache2/sites-enabled/000-default.conf and make the following changes

1.  <VirtualHost *: 80> should be changed as <VirtualHost *: 85>
2. change the DocumentRoot from /var/www/html  to /var/www/html_apache

Open folder /var/www/html and CUT and paste index.html to /var/www/html_apache

If you are unable to save changes to files and get a permission error, run your text editor as sudo. If you get errors while copy-pasting index files run the file manager as root. In Lubuntu you can open a terminal and type sudo pcmanfm

Also, create an index.html inside /var/www/html_apache/test1 and include some text so that you know the response is from apache.

Restart apache with the command 

sudo service apache2 restart

Now opening Localhost should point to Nginx welcome page and localhost:85 should show the apache welcome page

Setting up Machine2:

In machine2 you can simply install apache2, no special changes are required. Get the local IP address of machine2 using the command ifconfig (mine was 192.168.1.5). Also, create an index.html inside /var/www/html/test2 and include some text 

I haven't set up PHP or database. You can do that if you want.

Setting up the reverse proxy:

To set up the reverse proxy, in machine1 you should edit the file /etc/nginx/sites-available/default (another approach will be to create a new conf file and linking it to sites-enabled, but I will be editing the default file). Find the block that says location / and at the end of that block add a new location as shown below

location /test1/ {
proxy_pass http://127.0.0.1:85;
}

This will pass any request of the form /test1/ to localhost:85/test1/ . For example, a request like /test1/apix will be forwarded to apache in machine1 as localhost:85/test1/apix

Similarly, for test2, you can just add the IP address of machine2 (if you are hosting it somewhere use its public IP) instead of the localhost address

location /test2/ {
proxy_pass http://192.168.1.5;
}

And that's it, you may also need to forward the ip address of the user to the apache servers. If not the address of the Nginx server will be recorded for each request. To do so add the headers as shown below

location /test1/ {
	proxy_set_header X-Real-IP  $remote_addr;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $host;
        proxy_pass http://127.0.0.1:85;
  }

You can also set the reverse proxy in such a way that all php files will be served from a particular server.

 
 

 

Editing Flutter Packages in Codemagic

There are times when we may have to edit packages downloaded by flutter, maybe to add a new functionality or customize the style etc. In this article we will take a look at how to use a package that we have edited manually, in codemagic.

Tools Required

• VNC viewer, which can be downloaded from here 

Setup Codemagic

While configuring the codemagic build steps. Under “Dependency caching”, check “Enable dependency caching” option and add the following path “$HOME/programs/flutter/.pub-cache/” you may add the the entire pubcache folder if you wish ( ie., $HOME/programs/flutter/.pub-cache). But this is not the $HOME/.pub-cache mentioned in the documentation. And now specify the build and publish options as usual. when you click on "Start new build" make sure you check on the "Enable remote access" checkbox and start the build process.

Connecting to the Machine

After the process starts in a few seconds codemagic will display the SSH command as well as the details required to connect via VNC. Open up the VNC Viewer program we downloaded earlier and in the field where it asks for "VNC server address" enter the ip address of the machine followed by the port number (for example, 192.xxx.xxx.xxx:10000, note the colon separating the ip and the port).And then click "connect to the host name option". It will show an insecure connection warning just click continue. Enter the username and password and click ok. And you will be able to see the desktop of the build machine.

Making the changes

Once you have access to the mac. Click on the finder app and click on “Go” and “go to folder” enter “/Users/builder/programs/flutter” and open that folder. Now click Left Alt+Shift+dot to show hidden folders. Then open the .pub-cache folder and move to the folder containing your package. You can right click the dart files and open them with Android Studio. Make your changes and save them (save changes often). I couldn’t find a way to send the files from local to remote machine but you should be able to upload it on some file-sharing site and download it there.

Note: In case the current build got finished before you made the changes. The changes won’t be reflected in the build. But since the folder will be cached for the next build. You can just build it again.

Limitation

• The connection will be terminated 20 minutes after the build completes. So we should complete everything before the 20 minute window runs out.

Running the IOS Simulator on a codemagic build machine

In this article we will take a look at how to do a preliminary check on our apps by running them in an IOS simulator via codemagic. Codemagic allows us to connect to the build machine through SSH and VNC. Here we will take a look at connecting to codemagic's build machine using a VNC program

Tools Required

• VNC viewer, which can be downloaded from here

Setup Codemagic

Complete the initial setup in codemagic (like specifying build and publish options) as usual. when you click on "Start new build" make sure you check on the "Enable remote access" checkbox and start the build process.

Connect to the Machine

After the process starts in a few seconds codemagic will display the SSH command as well as the details required to connect via VNC. Open up the VNC Viewer program downloaded earlier and in the field where it asks for "VNC server address" enter the ip address of the machine followed by the port number (for example, 192.xxx.xxx.xxx:18360, note the colon separating the ip and the port). And then click "connect to the host name option". It will show an insecure connection warning just click continue. Enter the username and password and click ok. And you will be able to see the desktop of the build machine.

Run the app in the simulator through Xcode

Use the Launchpad app to open Xcode (There will be multiple versions available, choose the one you used to build). Click on "open another project" and browse to the following location "/Users/builder/clone/ios" . Select the iPhone version for the simulator and click on the run button.

Limitation

• The connection will be terminated 20 minutes after the build completes. So we should complete everything before the 20 minute window runs out. 

Build a flutter app for IOS using codemagic - Part 2

In this article we will see how to build a flutter app for IOS using codemagic if we had already generated a key using a mac. Except for the first step all other steps are similar to my previous article

Step 1: Get the certificates from mac

In your mac open Keychain Access. Locate the key that you had used to sign the app previously. Select that along with the private key and right click “Export” select file format as .p12 and save. Enter a password and verification for the file and click ok. Enter your mac password and click ok and save your file. Once you have the p12 file file you can use it from other computers with different os.

Step 2: Add the identifiers

On the apple developer page. Click on "Identifiers" and then click the Plus sign near the heading that says Identifiers, select “App IDs” and continue. Enter the name of your app in Description and your apps Bundle ID in the respective field. Click Continue. Make sure the bundle Id match with your apps bundle ID in "\ ios\ Runner.xcodeproj\ project.pbxproj".

Step 3: Add test Devices (optional) 

On the apple developer page. Click on "Devices" and then click the Plus sign near the heading that says Devices, Give a name to the device so that you can identify it. And enter the UDID of the device. Click Continue and register. Repeat for all devices you may want to distribute the app. 

Step 4: Add distribution profiles

On the apple developer page. Click on "Profiles" and then click the Plus sign near the heading that says Profiles, Since we generated the sign key for Distribution. Select an option under “Distribution”,  I will be using "Ad Hoc" so choose it and click continue.

• Under App ID select the name you created for Step2 and click continue.
• In the next step select the key you created previously in Step1 and click continue
• In the Select Devices step select all the devices you want to install the app on. (Probably the ones created in Step3). Click continue
• Enter a name for your profile under Provisioning Profile Name. Click Generate and Download the mobileprovision file

Note: You can generate multiple provisions for the same app. Maybe an 'App Store' provision in addition to the 'Ad Hoc' provision

 Step 5: Configuring codemagic

I have assumed that you have already linked your git repo with codemagic and the project you want to build is open

• Expand Build, Set the necessary version for Flutter and Xcode. Under 'Build for platforms' uncheck the platforms you don’t need and change the Mode to release. Click save
• Expand Publish,

1. Under 'iOS code signing' choose Manual.
2. Under 'Code signing certificate' click choose a file and browse and select the .p12 file we created in Step1
3. Enter the password you had given to the file in the 'Certificate password' field
4.  Under 'Provisioning profiles' upload the Profiles you generated in Step4. Save it

Click on 'Start new build'

Step 6: Distributing the app

Once the build finishes you will get an email with the .ipa file and a .app file. You can upload the .ipa file to sites like diawi and share the link with users whose mobile numbers are registered in the app.

Limitations

• By moving away from mac. We lose the ability to use the Xcode simulator. So we can only test the app after a user/test-user installs it in a real device. A workaround can be found here
• Since we are building on codemagic. We can't use custom edited packages. ie., in case we had manually edited a package in our local environment those changes can't be expected in the codemagic build since it will fetch from the original package repository. A workaround can be found here