Bobby Doshi | RSS Feed

The Deli Dilemma: Perspective on Menu Accessibility and Customer Experience

doshi.bobby@outlook.com — Sat, 29 Jul 2023 00:00:00 GMT

Inside Northeastern University lies Wollaston's Market, a deli that serves as a microcosm of student life and diversity. My tenure as a deli staff member for the past six months has granted me a front-row seat to the intricate dynamics between a menu's design and the customer experience it shapes.

A Tale of Text and Trials

Gazing out from my position behind the counter, it's become routine to see patrons squinting at the digital menu that stands behind us. It's not uncommon to see the text proving challenging for those with glasses or more mature eyes—a digital divide that necessitates the alternative of a paper menu for better accessibility.

The Quest for Greens

The menu’s options for vegetarian or meat-free sandwiches often get lost in translation, leading to a default choice: the 'Veggie Delight.' Amidst the grill's sizzle, a recurring question arises from our patrons: "What else is vegetarian here?" It's a signal that the vegetarian options, though present, are lost in a menu.

The Rush Hour Riddle

During the deli's peak hours, the decision-making process slows, and queues lengthen as customers navigate the menu.

A Sandwich on a Budget

The wallet’s weight is a universal consideration—students especially seek out offerings that promise a reprieve to their budget. They scan the menu with a discerning eye, searching for that sweet spot where cost and craving meet.

Towards a Solution

These observations pivot towards a crucial inquiry: How can I synthesize these insights into a menu design that speaks to the varied needs of Northeastern's students and staff? My aim is to leverage UI/UX design principles to create a menu that is legible, efficient, and sensitive to dietary and fiscal preferences.

Moving forward, I seek to marry functionality with inclusivity, transforming the menu into a tool that not just lists options but facilitates a positive, seamless customer experience.

Surveying for Customer Insight

I prepared a set of questions. My goal was not merely to confirm my observations but to unearth the depth of our customers' needs and preferences. With each sandwich crafted and wrapped, a new piece of the puzzle was shared by the one who would savor it.

How would you rate the ease of navigating our digital menu today?
Was the text on our digital menu clear and easy to read?
How quickly were you able to decide what to order from the menu?
Did you find our menu's layout to be intuitive and organized?
Were the descriptions of our sandwiches helpful in making your decision?
Can you tell me about any challenges or frustrations you experienced while looking at the menu?
If you have specific dietary preferences, was it easy to find suitable options on our menu?
Do you have any suggestions on how I could improve our menu to serve you better?

As my survey unfolded, the feedback gathered began to sketch a vivid portrait of customer experience. While some findings echoed my initial observations, others unveiled new layers of our patrons' needs. Here are some of the shared sentiments:

Navigational Challenges: Patrons expressed a common frustration: "Finding what I want can be a real scavenger hunt. The menu keeps shifting between screens, and items aren't where I last saw them. It'd help if each menu item had a fixed spot."
Clarity in Dietary Options: The plight of non-meat eaters was highlighted, with responses like "I'm vegetarian, and it's a hassle to hunt down the non-meat choices. Sometimes I end up just picking the same thing because it's too hard to distinguish the vegetarian options from the rest."
Bread Variety Confusion: Customers voiced surprise over the bread choice, suggesting "Maybe include bread types in the item names or descriptions? I was surprised when my sandwich came on a ciabatta; I expected the usual sub bread."
Meat Preference Specificity: Meat lovers yearned for clarity, noting, "I specifically look for chicken sandwiches, so I read the descriptions closely. It would be great if the menu highlighted the main protein so I could spot my favorites faster."
Price Visibility: Several responses centered around financial concerns: "I'm always on a budget, so I want to see prices at a glance. Sometimes, it takes me a while to locate the price on the screen."
Speed of Selection: During peak periods, the pressure mounts, as one customer remarked, "During busy times, I feel rushed to choose because I don’t want to hold up the line. A more streamlined menu might speed things up."

These generalized answers not only aligned with my own observations but also added depth to my understanding of the customer experience at Wollaston's Market.

Implementing UI/UX Principles

After gathering customer feedback and understanding the crucial aspects that needed improvement, I embarked on redesigning the menu with a focus on usability and accessibility. Here's how I applied various UI/UX principles to the new menu design:

New Minimalist Digital Menu Signage

Embracing Hick's Law

I've grouped sandwiches by price in ascending order and sorted them alphabetically within each group to simplify the decision-making process. This method aligns with Hick's Law, suggesting that limiting choices can speed up and ease the selection process for my customers.

Law of Similarity and Color Theory

Utilizing the Gestalt principle of similarity, I've highlighted vegetarian options in green. This strategy aids customers in quickly spotting meat-free choices while incorporating color theory principles, where green signifies plant-based and freshness.

Enhancing Visual Hierarchy

I chose a bold font for sandwich names to establish a clear visual hierarchy. This allows patrons to swiftly identify their sandwich of choice and navigate the menu with greater ease.

In addition to these improvements, I made a deliberate choice regarding the background color and font used in the new menu design. The background was set to #FEFEFE instead of a stark #FFFFFF. This decision was made considering the display technology—a LED TV—which has a tendency to overemphasize bright colors, making pure white backgrounds potentially glaring and harder on the eyes. The slightly off-white #FeFeFe offers a softer visual experience, reducing eye strain for customers.

The font selected for the menu is LATO, known for its readability and modern aesthetic. LATO balances well between being highly legible for all ages and adding a contemporary look to the menu, aligning with the deli's forward-thinking brand identity. This choice underscores the importance of readability and aesthetics in enhancing user experience, ensuring that the menu is not only easy to read but also visually appealing.

Applying the Law of Proximity

In the revamped menu, I've placed items within the same price category in closer proximity to each other. This arrangement indicates a relationship between the items based on price, embracing the Law of Proximity.

Improving Accessibility

To better serve customers with visual impairments, the blue background was removed to amplify the contrast. This improvement makes the text stand out more, ensuring readability from various distances.

Focusing on Consistency

I've adopted a uniform approach to describing menu items: starting with the type of bread, followed by the main ingredient, cheese, vegetables, and dressings. This consistent pattern reduces the cognitive load for customers as they scan through the menu options.

The new design remains on a single screen without shifting, addressing navigational challenges. Also, by improving the layout to follow an “F Pattern,” we align with natural reading patterns, from left to right, and then top to bottom, enhancing the speed of selection during rush hours.

After addressing the immediate needs of our customers through a thoughtful redesign of the menu, the next phase focused on the backend—specifically, the process by which the deli team updates and deploys this menu. The original system, based on a MERN stack, was a complex beast. It required a server to be running in the background and any changes to the UI necessitated a series of convoluted steps. For a team primarily composed of individuals without a deep technical background, this setup was far from ideal. It was clear that a more user-friendly approach was needed to ensure that our menu could evolve alongside our patrons' tastes and preferences without the burden of technical overhead.

The Technical Redesign

The solution came in the form of a Python script, designed to democratize the process of menu updates. This script, leveraging the simplicity and power of Python along with libraries like Pandas for data manipulation and BeautifulSoup for HTML template modification, provided a streamlined pathway from menu update to deployment.

File Structure Comparison

A visual representation can illustrate the contrast between the previous MERN stack implementation and the current Python-based solution. Here's a comparative view:

MERN Stack Complexity

Old MERN Stack File Structure

The MERN stack required multiple nested directories, separate CSV files for different menus, and depended on a constantly running server for menu display. The intricacy is evident in the folder hierarchy and necessitates detailed technical knowledge for any update or maintenance.

Streamlined Python Approach

New Python-based File Structure

In stark contrast, the Python-based approach consolidates the menu data into a single CSV file within a 'menu' directory. The absence of a complex folder structure signifies a simplified process, whereby a single batch file execution updates the entire menu.

The Departure from MERN

While the MERN stack solution offered robustness and scalability, it was an overengineered solution for the needs. The requirement to keep a server running just to display a menu was not only resource-intensive but also introduced unnecessary complexity for staff. The pivot to a more straightforward, serverless approach meant we could sidestep these issues, making menu management accessible to all staff members regardless of their technical know-how.

The Process Unpacked

Data Input and Organization: At the heart of my new system is a CSV file—a simple, editable format familiar to anyone who has used spreadsheet software. This file contains all the menu items, their descriptions, prices, and other relevant details. When updates are needed, be it a new sandwich or a change in price, staff members directly edit this CSV, no complex coding required.
Automated Sorting and HTML Creation: Upon updating the CSV file, the next step involves running the Python script. This script first sorts the menu items based on predefined criteria (dietary flags, price, etc.), ensuring the menu's logical organization. It then generates HTML files for each menu section by populating a base template with the sorted data. This automated process ensures consistency in presentation and adherence to our UI/UX principles.
Simplified Deployment: The final step in the process is launching the menu. This is achieved through a batch file (execute.bat), which, when double-clicked, runs the Python script. The script's execution updates the menu and automatically opens it in a web browser, ready to be viewed by staff and customers alike. This step remains constant whether or not the CSV file has been updated, ensuring a seamless experience for the user.

From a Non-Technical Perspective

For the staff, this new system transformed what was once a daunting task into a simple, two-step process. If the menu needs updating, they edit a single CSV file—a task as simple as updating a spreadsheet. Then, with a double click on the execute.bat file, the updated menu is automatically sorted, formatted, and displayed in a browser. This process doesn't just save time; it eliminates the fear of making mistakes.

Conclusion and Next Steps

Go and have a look if you are on campus!

Through this journey of transforming the deli menu at Northeastern University's Wollaston's Market, I've merged my background in software engineering and data science with the dynamic world of UI/UX design. This project has been a profound learning experience, showcasing how UI/UX principles can significantly elevate customer experience.

As an individual with a penchant for problem-solving and a deep appreciation for user-centric design, this endeavor has enriched my understanding of how technology and design intersect to improve everyday experiences. I have simplified the menu management process and made it more accessible for the staff, leveraging Python and its powerful libraries to streamline updates and deployment.

However, the journey towards perfection is ongoing. One potential area of improvement lies in better categorization and the inclusion of pictorial representations of top-selling items. This strategy could enhance the visual appeal of the menu and aid in decision-making, particularly for new customers. Balancing the comprehensive display of the menu with the limited space available presents a challenge that requires thoughtful consideration and innovative solutions.

For those interested in the technical details or considering implementing a similar solution, I have made the source code available on GitHub. I encourage you to explore, adapt, and contribute to the project.

Complete code is available on my Github repo 👇

https://github.com/BobErgot/Digital-Signage-Menu-Board

What were the odds of Jack Dawson surviving the Titanic?

doshi.bobby@outlook.com — Tue, 16 Aug 2022 00:00:00 GMT

This article was originally published on Medium and can be found here

While browsing through the Kaggle, I noticed the famous Titanic dataset competition to predict which passengers would have survived. Having watched the American romance movie Titanic, it piqued my interest to find the extent to which the survival of passengers was dependent on age, gender, and social status.

Introduction to the chosen techniques

A decision tree is a decision support tool that uses a tree-like model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility. — Wikipedia

Decision Trees help in understanding the data. Given their transparency and relatively low computational cost, Decision Trees are also useful for exploring the data before applying other algorithms. They help check the quality of engineered features and identify the most relevant ones by visualizing the resulting tree.

The main downsides of Decision Trees are their tendency to over-fit, inability to grasp relationships between features, and the use of greedy learning algorithms. Finding the best tree depth helps to resolve the issue of overfitting generally observed in such models.

Introduction of the dataset

Let’s load the data and get an overview. Data set can be found on here.

# Imports needed for the script
import numpy as np
import pandas as pd
import re
import xgboost as xgb
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline

import plotly.offline as py
py.init_notebook_mode(connected=True)
import plotly.graph_objs as go
import plotly.tools as tls

from sklearn import tree
from sklearn.metrics import accuracy_score
from sklearn.model_selection import KFold
from sklearn.model_selection import cross_val_score
from IPython.display import Image as PImage
from subprocess import check_call
from PIL import Image, ImageDraw, ImageFont
from sklearn.preprocessing import LabelEncoder
from sklearn_pandas import DataFrameMapper
from sklearn.preprocessing import OneHotEncoder

# Loading the data
train = pd.read_csv('train.csv')
test = pd.read_csv('test.csv')

# Showing overview of the train dataset
train.head(5)

Overview of the train dataset

#Check the dimensions of the dataset
train.shape

Dimensions of the dataset

# Check for null values in train and test data
print("Null values in the train data - ")
display(train.isnull().sum())

print("\nNull values in the test data - ")
display(test.isnull().sum())

Null values in train and test data

def concat_df(train, test):
    return pd.concat([train, test], sort = True).reset_index(drop=True)

full_data_df = concat_df(train, test)

#Check the percentage of missing data in entire data for column Age and Cabin
print("Percentage of missing values for age variable in entire data - ")
display(round(full_data_df['Age'].isnull().sum()/len(full_data_df)*100.0))

print("\nPercentage of missing values for cabin variable in entire data - ")
display(round(full_data_df['Cabin'].isnull().sum()/len(full_data_df)*100.0))

Percentage of missing data in entire data for column Age and Cabin

The dataset consists of 891 rows and 12 columns:

PassengerId
Survived: Only present in train dataset and is already in binary format
```
0 = No
1 = Yes
```
Pclass: A proxy for socio-economic status (SES)
```
1st = Upper
2nd = Middle
3rd = Lower 
```
Name
Sex
Age: Fractional if less than 1. If the age is estimated, it is in the form of xx.5. Null values are observed in train and test data. 20% of data is missing in the entire dataset

Sib Sp: The dataset defines family relations

Sibling = brother, sister, stepbrother, stepsister
Spouse = husband, wife (mistresses and fiancés were ignored)

Parch: The dataset defines family relations

Parent = mother, father
Child = daughter, son, stepdaughter, stepson
*Some children traveled only with a nanny, therefore parch=0

Ticket: Have many unique values with numbers and text
Fare: Null value observed in test data
Cabin: Null values observed in train and test data. 77% of data is missing in the entire dataset

Embarked: Port of Embarkation

C = Cherbourg
Q = Queenstown
S = Southampton

Null value observed in train data

Input encoding/input representation (How and why?)

Dealing with missing values as observed in “Introduction of the dataset”

Age

plt.hist(train.Age, edgecolor='black')
plt.xlabel('Age')
plt.ylabel('count')
plt.show()

As observed in the above section 20% of the age variable is missing in the entire data set. However, one hypothesis is that age played a vital role in the survival of passengers. Hence instead of deleting the rows with missing values, the median should be a good choice for substitution based on the right-skewed distribution of the age graph.

One hypothesis is that the median age differs for the passenger classes and sex.

#Check whether the thesis holds true
display(train.groupby(['Pclass', 'Sex'])['Age'].median())

The hypothesis may be true as passengers with higher socioeconomic status are older on average which can be attributed to the generalization that older people would have advanced further in their professional careers. At the same time, there is a uniform pattern for median difference for sex. Females tend to be younger.

Hence, median values are used for age based on the “Pclass” and “Sex” variables

full_data_df['Age'] = full_data_df.groupby(['Pclass', 'Sex'])['Age'].apply(lambda x: x.fillna(x.median()))

Fare

full_data_df.loc[full_data_df['Fare'].isnull()]

There is one missing fare value in the whole data set. Mr. Thomas was in passenger class 3, traveled alone, and embarked in Southampton. We will take other cases from people in this category and replace the missing Fare with the median of this group.

passenger_1043 = full_data_df[(full_data_df['Pclass']==3) & (full_data_df['SibSp'] == 0) & (full_data_df['Embarked'] == 'S')]['Fare'].median()
full_data_df.loc[full_data_df['Fare'].isnull(), 'Fare'] = passenger_1043
print(passenger_1043)

7.925 --> Median Fare

Embarked

full_data_df.loc[full_data_df['Embarked'].isnull()]

For train dataset, there are only 2 missing values. We can simply input the mode or apply a similar procedure as that of fare. However, quick research on Google engine led us to https://www.encyclopedia-titanica.org/titanic-survivor/ which showed both of them embarking from Southampton.

full_data_df.loc[full_data_df['Embarked'].isnull(), 'Embarked'] = 'S'

All the missing values are dealt with for variables “age”, “fare” and “embarked”. For now, “cabin” variable is ignored as missing data is 77% in the entire dataset.

display(full_data_df.isnull().sum())

##Feature Engineering

Creating new feature

Title

full_data_df['Title'] = full_data_df.Name.str.extract(r'([A-Za-z]+)\.', expand=False)
full_data_df.Title.value_counts()

There are 18 different titles in our data set.

The common titles are (Mr/Miss/Mrs/Master). Some of the titles (Ms/Lady/Sir) can be grouped into common titles.

The remaining unclassified titles, with less than 10 cases will be assigned to the category “others”.

full_data_df['Title'].replace(['Ms', 'Mlle', 'Mme'], 'Miss', inplace=True)
full_data_df['Title'].replace(['Lady'], 'Mrs', inplace=True)
full_data_df['Title'].replace(['Sir', 'Rev'], 'Mr', inplace=True)

common_title = (full_data_df['Title'].value_counts() < 10)
full_data_df['Title'] = full_data_df['Title'].apply(lambda x: 'Others' if common_title.loc[x] == True else x)
full_data_df.groupby('Title')['Title'].count()

Family Size

SibSp defines how many siblings and spouses a passenger had and parch how many parents and children. We can summarize these variables into the family size.

#add one to include the passenger itself
full_data_df['FamilySize']= full_data_df['SibSp'] + full_data_df['Parch'] + 1

Binning

Binning is a way to group several more or less continuous values into a smaller number of “bins”. It makes the model more robust and avoids over-fitting.

Age

full_data_df.boxplot(column=['Age'], figsize=(7,7))

With cut, the bins are formed based on the values of the Age variable, regardless of how many cases fall into a category. This is done so that the outliers do not irritate the algorithm.

full_data_df['Age'] = pd.cut(full_data_df['Age'].astype(int),  5)
full_data_df['Age'].value_counts()

#Encoding the bins into number for modelling
label = LabelEncoder()
full_data_df['Age'] = label.fit_transform(full_data_df['Age'])full_data_df['Age'].value_counts()

Fare

full_data_df.boxplot(column=['Fare'], figsize=(7,7))

The range of values is much higher for fare compared to age. With qcut we decompose Fare distribution so that there is the same number of cases in each category/bin.

full_data_df['Fare'] = pd.qcut(full_data_df['Fare'],  5)
full_data_df['Fare'].value_counts()

#Encoding the bins into number for modelling
full_data_df['Fare'] = label.fit_transform(full_data_df['Fare'])full_data_df['Age'].value_counts()

Labeling and One Hot Encoding

Label Encoding maps non-numerical values to numbers. This is because most algorithms cannot do anything with strings, so the variables are often recoded before modeling.

A one-hot encoding is a representation of categorical variables as binary vectors. This first requires that the categorical values be mapped to integer values. Then, each integer value is represented as a binary vector that is all zero values except the index of the integer, which is marked with a 1.

Many algorithms assume that there is a logical sequence within a column. However, this is not always expressed by the numerical ratio. Therefore it is needed to one-hot encoding the variables afterward.

#Labeling
non_numeric_variables = ['Embarked', 'Sex', 'Title']

for variable in non_numeric_variables:
  full_data_df[variable] = label.fit_transform(full_data_df[variable])
  
#One hot Encoding
variables = ['Pclass', 'Sex', 'Embarked', 'Title', 'FamilySize', 'Age', 'Fare']

encoded_variables = []

for variable in variables:
  encoded_feature = OneHotEncoder().fit_transform(full_data_df[variable].values.reshape(-1, 1)).toarray()
  num = full_data_df[variable].nunique()
  cols = ['{}_{}'.format(variable, num) for num in range(1, num + 1)]
  encoded_df = pd.DataFrame(encoded_feature, columns=cols)
  encoded_df.index = full_data_df.index
  encoded_variables.append(encoded_df)

full_data_df = pd.concat([full_data_df, *encoded_variables], axis=1)
full_data_df.head(2)

Coding for the implementation

Split the dataset into train and test data

def divide_df(full_data_df):
  return full_data_df.loc[:890], full_data_df.loc[891:].drop(['Survived'], axis=1)


train_data, test_data = divide_df(full_data_df)

Drop columns for which new columns are created through one hot encoding and redundant columns like cabin, passengerid, name, etc.

drop_cols_train = ['Embarked', 'FamilySize', 'Age','Fare' ,'Name','Parch', 'PassengerId', 'Pclass', 'Sex', 'SibSp', 'Title', 'Ticket', 'Cabin']
train_data = train_data.drop(columns=drop_cols)
test_data = test_data.drop(columns=drop_cols)

Finding an optimal depth by cross validation method to avoid overfitting

cross_validation_folds = KFold(n_splits=10)
accuracies = list()
max_attributes = len(list(test_data))
depth_range = range(1, max_attributes + 1)

# Testing max_depths from 1 to max attributes
for depth in depth_range:
    fold_accuracy = []
    tree_model = tree.DecisionTreeClassifier(max_depth = depth)
    for train_fold, valid_fold in cross_validation_folds.split(train_data):
        f_train = train_data.loc[train_fold]
        f_valid = train_data.loc[valid_fold]

        model = tree_model.fit(X = f_train.drop(['Survived'], axis=1), y = f_train["Survived"])
        valid_acc = model.score(X = f_valid.drop(['Survived'], axis=1), y = f_valid["Survived"])
        fold_accuracy.append(valid_acc)

    avg = sum(fold_accuracy)/len(fold_accuracy)
    accuracies.append(avg)
    
df = pd.DataFrame({"Max Depth": depth_range, "Average Accuracy": accuracies})
df = df[["Max Depth", "Average Accuracy"]]
print(df.to_string(index=False))

The best max_depth parameter seems therefore to be 5 which has 82.82% average accuracy across the 32 folds)

Hence, 5 is used as the max_depth parameter for the final model.

# Create arrays of train, test and target dataframes to feed into our models
y_train = train_data['Survived']
x_train = train_data.drop(['Survived'], axis=1).values 
x_test = test_data.values

# Create Decision Tree with max_depth = 5
decision_tree = tree.DecisionTreeClassifier(max_depth = 5)
decision_tree.fit(x_train, y_train)

# Export our trained model as a .dot file
with open("tree.dot", 'w') as f:
     f = tree.export_graphviz(decision_tree,
                              out_file=f,
                              max_depth = 5,
                              impurity = True,
                              feature_names = list(train_data.drop(['Survived'], axis=1)),
                              class_names = ['Died', 'Survived'],
                              rounded = True,
                              filled= True )
        
#Convert .dot to .png to display
check_call(['dot','-Tpng','tree.dot','-o','tree.png'])

img = Image.open("tree.png")
draw = ImageDraw.Draw(img)
draw.text((10, 0), (0,0,255))
img.save('sample-out.png')
PImage("sample-out.png")

Decision Tree

decision_tree_accuracy = round(decision_tree.score(x_train, y_train) * 100, 2)
decision_tree_accuracy

#Decision Tree Accuracy
84.18

Analysis of Results

The decision tree achieved an accuracy of 84.18% across the training dataset.

Graphical Tree Representation:

The first line of each node except the leaf node shows the splitting condition in the form “feature <= value”.
Next Gini Impurity of the node, is a measurement used to build Decision Trees to determine how the features of a dataset should split nodes to form the tree
Samples is the number of observations contained in the node.
Value shows the class distribution of the samples [died count, survived count].
Class corresponds to the predominant class of each node

Dictionary of labels:

{‘Master’: Title_1, ‘Miss’: Title_2, ‘Mr’: Title_3, ‘Mrs’: Title_4, ‘Others’: Title_5}
{1: Pclass_1, 2: Pclass_2, 3: Pclass_3}
{Female: Sex_1, Male: Sex_2}
Age and Fare have been mentioned above

Model can be summarized with following rules:

If it doesn’t include “Mr” Title, and Passenger class is not 3rd class, then we classify it as survived. (all the branches on the left side of the tree lead to a blue node)
If our observation includes “Mr” Title, and Passenger class is not 1st class then we classify it as not survived (has the highest casualties. Out of the 549 casualties in test data, 372 casualties belong to this class)
If our observation belongs to age bracket of 16 and above and has “Mr” title then the mortality rate increases

With the same observation if we consider Jack Dawson from Titanic movie, age 20 years, a poor artist, based on the above rules he fell right into the class of non-survivals.

A strong correlation can be observed with respect to survival and gender, age & social status class.

Automate LinkedIn Connection Request using Python and Selenium

doshi.bobby@outlook.com — Fri, 12 Aug 2022 00:00:00 GMT

This article was originally published on Medium and can be found here

What is LinkedIn?

It is a social media for professionals to showcase their achievements, for some to network and find jobs, and for others to use it as Facebook.

Why are we here?

Automation stems from the desire for efficiency. So, here is a short story.

Harry, my friend, posts on the WhatsApp group a google excel sheet in which all have to enter their LinkedIn profile URL so we can connect. Sounds great! It doesn’t matter if you know people, but “Gotta Catch ’Em All”.

I wake up in the morning and check the sheet and there are 246 entries. My actions looked as follows -

Click on Google Sheet tab > Click on Profile URL > Click “Connect” once the LinkedIn page loads > Close LinkedIn tab > Repeat.

Let’s automate it.

Scripting Requirements

Setting up environment

Install Python from here
Create a virtual environment “env” under the project directory “automate_linkedin”
```
python -m venv \base_path\automate_linkedin\env
```
Install Selenium and Pandas
```
pip install selenium
pip install pandas
```
Download geckodriver from here and extract it under the project directory

Crack your knuckles to Code

Get the data from csv file and generate a list of LinkedIn profile URL

linkedin_database = pandas.read_csv("linkedin_database.csv")
profile_url_list = list(linkedin_database.iloc[0:, 2])

where our CSV is in the format

csv structure

Provide the path for geckodriver for the Firefox browser and start the session

service = Service('geckodriver-v0.31.0-win64\\geckodriver.exe')
browser = webdriver.Firefox(service=service)

Automate Login
- Load the base login URL of LinkedIn
```
url = "https://www.linkedin.com/"
browser.get(url)
```
- Inspect web page and find ID for HTML element for username text field and password text field
- Get element by ID discovered above using selenium find_element method
```
username = browser.find_element(By.ID,"session_key")
password = browser.find_element(By.ID,"session_password")
```
- Next, we are sending keys, this is similar to entering keys using your keyboard. For this we use selenium send_keys method
```
username.send_keys("<Enter your email id>")
password.send_keys("<Enter your password>")
```
- Finally, inspect web page and find class_name for HTML element for submit button, get that element using class_name and click it using click method
```
browser.find_element(By.CLASS_NAME,"sign-in-form__submit-button").click()
```

Sending Connection Requests

Fetch each profile URL, check if it is a valid URL and then load the URL and wait for a few seconds till the web page is completely loaded

for profile_url in profile_url_list:
      print(profile_url)
      if not profile_url or pandas.isnull(profile_url):
            print("Empty String\n")
            continue
      try:
            browser.get(profile_url)
      except InvalidArgumentException:
            print("Bad profile URL\n")
            continue
      sleep(10)

Understanding different scenarios we may encounter while sending a connection request
- Scenario 1: “Connect” button is on screen itself
  
  “Connect” button is visible on screen
- Scenario 2: “Connect” button is under “More” dropdown
  
  “Connect” action under “More” dropdown
- Scenario 3: You are already connected with that user or you had previously sent a request which is in “Pending” mode
  
  Hence, we have to handle Scenarios 1 and 2 and ignore any instances of 3rd.

Finding the XPath to the “Connect button”. More on XPath can be found here.

Scenario 1: For this, we only need to find XPath to “Connect” button on the screen, get element by XPath and then click on it

def check_exists_by_xpath(browser,xpath):
  try:
      browser.find_element(By.XPATH,xpath)
  except NoSuchElementException:
      return False
  return True

xpath="//main[@id='main']/section[1]/div[2]/div[3]/div[1]/button[1]/span[contains(.,'Connect')]"
connect_xpath="//main[@id='main']/section[1]/div[2]/div[3]/div[1]/button[1]"

if check_exists_by_xpath(browser,xpath):
  browser.find_element(By.XPATH,connect_xpath).click()
  check_popup(browser)
  print("Connected\n")

Scenario 2: Here we first have to find XPath to “More” dropdown > get that element > click on it > find Xpath to “Connect” button under it > get that element > click on it

more_xpath="//main[@id='main']/section[1]/div[2]/div[3]/div[1]//button[contains(@aria-label, 'More actions')]"
more_connect_xpath="//main[@id='main']/section[1]/div[2]/div[3]/div[1]//button[contains(@aria-label, 'More actions')]/parent::node()/div/div/ul//div/span[text()='Connect']"
if check_exists_by_xpath(browser, more_xpath):
  browser.find_element(By.XPATH,more_xpath).click()
  if check_exists_by_xpath(browser, more_connect_xpath):
      browser.find_element(By.XPATH,more_connect_xpath).click()
      check_popup(browser)
      print("Connected\n")
  else:
      print("No connection option under More button\n")

Handling the different pop-ups post clicking “Connect” Let’s discover the different pop-ups we get on clicking “Connect”

Pop-up 1:

“Customize Invitation” pop-up

def check_exists(element,browser,sel):
  try:
      browser.find_element(element,sel)
  except NoSuchElementException:
      return False
  return True
  if check_exists(By.CSS_SELECTOR,browser,'[aria-label="Send now"]'):
      browser.find_element(By.CSS_SELECTOR,'[aria-label="Send now"]').click

In the above code snippet, we are trying to find an identifier for “Send” button which here is CSS_SELECTOR ‘[aria-label=”Send now”]’ for that element and then clicking on it.

Pop-up 2:

“How do you know” pop-up
```
if check_exists(By.CSS_SELECTOR,browser,'[aria-label="Other"]'):
  browser.find_element(By.CSS_SELECTOR,'[aria-label="Other"]').click()
  browser.find_element(By.CSS_SELECTOR,'[aria-label="Connect"]').click()
```
In the above code snippet, we are trying to find an identifier for “Other” button which here is CSS_SELECTOR ‘[aria-label=”Other”]’ and then for “Connect” which is CSS_SELECTOR ‘[aria-label=”Connect”]’. “Connect” will only get activated once “Other” or any one of the options is selected.

Pop-up 3:

“Prefers to be followed” pop-up

if check_exists(By.CSS_SELECTOR,browser,'[aria-label="Connect"]'):
  browser.find_element(By.CSS_SELECTOR,'[aria-label="Connect"]').click()

In the above code snippet, we are trying to find an identifier for “Connect” which is CSS_SELECTOR ‘[aria-label=”Connect”]’ and then clicking on it.

These pop-ups may come up one after the another and hence we create some kind of recursive method till all pop-ups are cleared.

def check_popup(browser):
  if check_exists(By.CSS_SELECTOR,browser,'[aria-label="Send now"]'):
      browser.find_element(By.CSS_SELECTOR,'[aria-label="Send now"]').click()
  elif check_exists(By.CSS_SELECTOR,browser,'[aria-label="Other"]'):
      browser.find_element(By.CSS_SELECTOR,'[aria-label="Other"]').click()
      browser.find_element(By.CSS_SELECTOR,'[aria-label="Connect"]').click()
      check_popup(browser)
  elif check_exists(By.CSS_SELECTOR,browser,'[aria-label="Connect"]'):
      browser.find_element(By.CSS_SELECTOR,'[aria-label="Connect"]').click()
      check_popup(browser)
  return

Now sit back and enjoy!!!

Note: This article serves as an educational tool on the use of selenium and does not encourage breaking of any LinkedIn rules and policies and as such should not be used in any form that harms anyone.

Complete code is available on my Github repo 👇

https://github.com/BobErgot/LinkedIn-Connection-Request-Bot

Innovating for Impact - Improving India’s Critical Suicide Prevention Services

doshi.bobby@outlook.com — Fri, 15 Jan 2021 00:00:00 GMT

Initiating Change

Around 2018, I became involved with Enroot Innovation Foundation, an innovation-first organization dedicated to addressing social issues through the power of technology. Enroot leverages design thinking and engineering innovation to create sustainable solutions for social change.

A pivotal moment occurred around World Suicide Prevention Day on September 10th, 2018. A simple Google search for “suicide” brought up Aasra, a feature integrated by Google to display prompts for users in distress. Similarly, Aasra appeared on Facebook, Quora, and WhatsApp and other platforms.

Google search prompt for suicide keyword

This prompted me to investigate further. Aasra, part of Befrienders Worldwide/Samaritans, a UK-based charity, is linked to a global network including Volunteer Emotional Support Helplines (VESH), Lifeline International, and the International Federation of Telephone Emergency Services (IFOTES). Established in the 1990s, Aasra was among the first services in India dedicated to suicide prevention.

In January 2006, Aasra was honored with the Maharashtra Ratna Gaurav Puraskar after ten years of high-quality service and significant efforts in suicide prevention, highlighting its vital role and impact.

Challenges

On deeper dive into Aasra’s services revealed growing negative feedback. Users reported issues such as non-responsiveness, difficulty reaching the helpline, service denial, or inadequate support from counselors. This was quite shocking for an organization known to address such issues, especially for a service addressing people at the most vulnerable times of their lives and could, in fact, drive them to take drastic measures.

Platforms like Quora were filled with negative reviews and accounts of service denial, indicating this problem extended beyond Aasra to other helplines as well.

User answer from Quora - 1

User answer from Quora - 2

Understanding the Problem

To address the root causes effectively, it was essential to understand the problem from the organization’s perspective. Rishabh facilitated my contact with Johnson Thomas, the founder of Aasra. During our discussion, I presented our findings, including various reviews and feedback about the helpline’s performance. Johnson shared the historical context and the evolution of Aasra, highlighting the significant challenges they faced.

Johnson explained that Aasra started in the 1990s with a small group of 36 individuals dedicated to providing emotional support through a helpline. Initially, it was a local initiative focused on Mumbai. However, over time, Aasra gained recognition and numerous awards, which led to its inclusion in distress-related search results and social media prompts across India. This expansion drastically increased the demand for their services, which their infrastructure struggled to support.

Aasra helpline in distress-related help shown on Facebook, WhatsApp & Quora among others

He mentioned that according to government data, nearly 10 million-20 million Indians (1%-2% of the population) suffered from severe mental disorders such as schizophrenia and bipolar disorder, and nearly 50 million (5% of the population) suffered from common mental disorders such as depression and anxiety by the end of 2005. Even a small fraction of these individuals seeking help from Aasra resulted in an overwhelming burden on their helpline.

Despite their efforts, Aasra remained self-funded and reliant on donations, which limited their ability to scale operations to meet the growing demand. Additionally, many of the contact numbers listed on various platforms were outdated and non-functional, further compounding the issue. Despite repeated attempts to update these listings, Johnson’s efforts were unsuccessful, leading to significant frustration among users seeking immediate help.

Outdated Numbers:
- Phone: 91-22-27546669
- Phone: 91-22-27546667

Rethinking, Engineering, Design, Execution

Phase 1: Website from a UI/UX Perspective

We started to rethink the problem. We did not have enough resources to either bring about changes in the law nor enough reach to raise huge amounts of funds. The only thing we knew was that thousands of people visited the Aasra website and called them daily.

So we started with just that. First, we did an analysis of the website. According to Google Analytics, 63% of the users that visited the website did so through mobile devices. However, the website was not device responsive, and the experience of navigating it through mobile was frustrating and not intuitive. This was another area of concern, especially for people at a vulnerable stage in their lives.

Aasra website old layout - desktop and mobile view

From the analysis of the website’s design, we identified several bad design components:

Non-Responsive Design: The website did not adapt to different screen sizes, making it difficult to navigate on mobile devices.
Cluttered Layout: The homepage was cluttered with too much text and information, overwhelming users.
Outdated Contact Information: The helpline numbers displayed were outdated and no longer in service.
Poor User Interface (UI): The design was not user-friendly, and important information was not highlighted effectively.
Inconsistent Navigation: The navigation menu was not consistent, making it hard for users to find what they were looking for.
Cold Imagery: The imagery used on the website was cold and unwelcoming, showing distress rather than providing a sense of warmth and openness that indicates support and help.

Archive link - Old Website before 2018

Engineering Improvements

To address these issues, we initiated the engineering improvements:

Responsive Web Design: We rebuilt the website using responsive web design techniques to ensure it adapted to various screen sizes, improving accessibility and user experience on mobile devices.
Backend Overhaul: We updated the backend infrastructure to ensure scalability and reliability, capable of handling increased traffic from across India. This included transitioning to a static website architecture and using content delivery networks (CDNs) to distribute content efficiently.
Enhanced Load Times: We optimized the website’s performance to reduce load times, crucial for users in distress seeking immediate help. This was achieved through code minification, image optimization, and leveraging browser caching.

Design Enhancements

Aasra website new layout - desktop and mobile view

We also focused on design enhancements:

Simplified Layout: We decluttered the homepage, prioritizing essential information and resources to make them easily accessible.
User-Friendly Navigation: We designed a consistent and intuitive navigation menu, allowing users to find information quickly and effortlessly.
Warm Imagery: We replaced cold, distressing images with warm, welcoming visuals that conveyed support and empathy.
Clear Call-to-Actions: We added clear call-to-action buttons to guide users to immediate help or resources.

Through this, we were able to transform Aasra’s digital presence, making it more responsive, accessible, and user-friendly for those in need. This iterative approach allowed us to continually refine and improve the website, ensuring it met the needs of its users effectively.

Neha, Nemil, Avi, Aditya, Anirudh, and Harsh helped me in the redesign of the website based on the feedback we had received from various mental health specialists and psychologists working in this field.

Archive link - New Website post 2019

Phase 2: Addressing the Burden on the Helpline

In the second phase of rethinking, engineering, design, and execution, we focused on addressing the burden on the helpline and ensuring that people seeking help got access to the right resource at the most critical time.

For this, I and the team at Enroot Mumbai discussed various ways we could manage the traffic to the helpline and ensured people get access to the most critical resources at the essential time. This issue was bigger than the organization as denial of service to people in a vulnerable state could also mean potential loss of human life. The entire purpose here was to save human lives.

Hence, the team decided to create the most exhaustive list of suicide prevention helpline directory for India. Each team member researched the various organizations working in this sector and personally verified that all the helplines collected were functioning and providing the services as promised by them.

Navigating through this highly fragmented social sector was tough. There were private organizations, non-profits, hospitals, and even government organizations providing various offerings, and many outdated resources. Many of the numbers and helplines listed on websites and organizational pages were no longer functional, or the institutions had dissolved due to growing burden and lack of resources. After reviewing over 350 helplines and resources, the team was able to come up with 52 helplines that were functioning (last verified as of 2021).

The main ideas behind this were:

If a helpline was busy or unavailable, users had the option to quickly go through the list and find another number.
The numbers were sorted by states so that users could access the closest resources.
Considering the numerous languages spoken in India, having helplines sorted by states made it possible for users to speak in their own mother tongue and convey their emotions easily.

From a design perspective, the numbers were also hyperlinked to a call-to-action, so users could simply click on the number, opening the dial pad on their mobile devices with the number already typed.

Another approach we implemented was to ensure that users landed on the Indian suicide helpline directory page to perform the call-to-action. This ensured that even if they wanted to call Aasra, the only place where the number was listed was on the helpline directory page. So, in case they were not able to reach the Aasra helpline, the same page provided them with ample alternative resources, thereby trying to minimize and manage distress in users.

Aasra - Suicide Prevention Helpline Directory

You can check the Helpline directory - Archive link

Ensuring social media platforms like Facebook and Google listed better alternatives when users keyed in distressed-based words was another critical aspect.

For this, I worked alongside Prem at Enroot Innovation Foundation to advocate for listing the iCALL Psychosocial Helpline as the primary resource instead of Aasra. The reason was that out of all the helplines we had researched, iCALL had a better structure and support system.

iCALL, based at Tata Institute of Social Sciences, Mumbai, and sponsored by Mariwala Health Initiative (MHI), had numerous partnerships and collaborations, including:

United Nations Population Fund
Government of Madhya Pradesh
Government of Gujarat
Government of Maharashtra
Larsen & Toubro Ltd.
Tata Consultancy Services
Dainik Bhaskar
nULTA Healthcare Services Pvt. Ltd.
Tata Starbucks
TRAINN Circle
Vishakha

Google search result for suicide post 2020 - After Local Guides suggested changes to Google

I would like to thank Prem especially for this as he helped me coordinate with other Local Guides to ensure that Google delisted Aasra and instead listed iCALL, thereby helping achieve another important step towards ensuring people reach the right resource.

Outcome (as of 2021)

The helpline page (http://www.aasra.info/helpline.html) has 3,086 backlinks as of 2021, data sourced from Alexa Internet. This page has been referred to by many news outlets and newspapers as a source of helpline information. It has also been included in other distress helpline directories and websites, further enhancing its visibility and reliability as a critical resource for those in need.

Contribute to This Project

The complete code for this project is available on my GitHub repository. We welcome contributions from the community to help develop and enhance the technical infrastructure for suicide prevention. Your support and collaboration can make a significant impact.

Complete code on GitHub

Bobby Doshi | RSS Feed

The Deli Dilemma: Perspective on Menu Accessibility and Customer Experience

A Tale of Text and Trials

The Quest for Greens

The Rush Hour Riddle

A Sandwich on a Budget

Towards a Solution

Surveying for Customer Insight

Implementing UI/UX Principles

Embracing Hick's Law

Law of Similarity and Color Theory

Enhancing Visual Hierarchy

Applying the Law of Proximity

Improving Accessibility

Focusing on Consistency

Menu Management: From Complexity to Simplicity

The Technical Redesign

File Structure Comparison

The Departure from MERN

The Process Unpacked

From a Non-Technical Perspective

Conclusion and Next Steps

What were the odds of Jack Dawson surviving the Titanic?

Introduction to the chosen techniques

Introduction of the dataset

Input encoding/input representation (How and why?)

Age

Fare

Embarked

Creating new feature

Title

Family Size

Binning

Age

Fare

Labeling and One Hot Encoding

Coding for the implementation

Analysis of Results

Automate LinkedIn Connection Request using Python and Selenium

What is LinkedIn?

Why are we here?

Scripting Requirements

Setting up environment

Crack your knuckles to Code

Scenario 1: “Connect” button is on screen itself

Scenario 2: “Connect” button is under “More” dropdown

Scenario 3: You are already connected with that user or you had previously sent a request which is in “Pending” mode

Pop-up 1:

Pop-up 2:

Pop-up 3:

Innovating for Impact - Improving India’s Critical Suicide Prevention Services

Initiating Change

Challenges

Understanding the Problem

Rethinking, Engineering, Design, Execution

Phase 1: Website from a UI/UX Perspective

Phase 2: Addressing the Burden on the Helpline

Phase 3: Ensuring Better Alternatives on Social Media Platforms

Outcome (as of 2021)

Contribute to This Project