hi can you hear me okay okay so welcome
back to the missing semester of your sea
acidification today are having a lecture
topic Opry which is it's gonna be some
miscellaneous combination of topics that
we the instructors find there are
interesting but none of them kind of war
on their own lecture because they're
kind of like certain topics that we just
want you to know about because they can
be again like really helpful and again
since we're not gonna to delve into a
lot of detail in the topics if you're
more interested all them just kind of
feel free to come and ask us questions
at the end or as we go over them in
lecture so the first thing I want to
talk or talk about these keyboard
remapping so by now you probably
realized that we have encouraged to use
the keyboards your main input method so
for example when we went into the
editors lecture there one of the main
ideas of BIM was using your keyboard as
much as possible so you have to kind of
rely go into the mouse because going to
the mouse is long and the thing is your
keyboard as with many things in your
computer is nothing kind of magical it
can be configured and it's worth
configuring because a lot of kind of the
defaults might not be optimal they kind
of the most simple modification that you
can do is just remove keys so one of the
things we alerted in the editor lecture
is the caps lock key is a really good
key because it's kind of driving the
home bro and it's kind of large and
there but it's not useful like you'll
probably realize that using users of
your tops lucky us it's like when would
you want to use that so you can just
remap your caps lock key to something
more useful as we mentioned like escape
you're a being used here or like control
your Emacs user a or useful dream
mappings like a lot of the upper row
like F function like the F keys or like
print screen you can remap them for
example to your media key so like when
you type print screen that you probably
don't have to do print screen that often
but you probably want to play or pause
your music and a lot of pretty much
every operating system has like some
tools that you
can use to configure this like they're
likely I'm not gonna go into the details
but there's like some of them listed in
the notes I'm limiting oh yeah the
another thing that you can do with
keyboard remapping so that you can do
more complex combinations you can have a
combination of keys mapped to some axiom
so for example I have keyboard mappings
that whenever I do ctrl enter I open a
new terminal window because the thing I
do Phegley often and by default magnet
is no key binding to do that or controls
if enter will open a new browser window
another operation that I do on a daily
basis so I don't have to like gonna grab
my mouse and go to Chrome do that and
you can also do room up in to perform
actions if you don't want to be typing
your password all the sorry your
personal your email or your password or
for example your MIT ID like you may not
remember it by heart
then you can just have like a keyboard
combination that will perform the action
of pasting that text lastly there like
more like right now it looks more than
like you just have to do some function
of this is the keys that you press and
this is the action that you happen but
actually there are more complex keyboard
combinations as you go through you kind
of learn so you can do kind of keyboard
sequences so for example when we were
dealing with timox timox there was kind
of this notion of off first you press
ctrl-a and then you all I control be
like you press some prefix and then some
other key and that means something a lot
of this software allows that so for
example in my image ever since I'm
reducing calf lock at all but every so
often I have to use when my undergrad of
some software that relied on capslock
for changing modes then I have I can
press shift five times in a row quickly
and then this this software that is in
the middle interpreting these comments
and prim up into some other will send a
single caps-lock command for that and
some more examples of that is that I
mentioned that you can use you
caps lock key like to map to escape or
control but you actually can dream up to
both so in my computer when ideas stop
the caps lock key that is interpreted as
an escape however if I press it and hold
it I can kind of this software can
understand the difference within quickly
pressing it and yes holding it for using
in combination with some other key and
then in that case is mapped to country
so a lot of these kind of more advanced
configuration is reported about a lot of
these tools and as I mentioned we have a
kind of a short list of good defaults
for these programs from kind of Windows
Mac OS and Linux any questions on this
topic ok now I'm going to cover
unrelated topic to keep our mapping
sleep we're going to see a lot of these
under later incisions and this lecture
and is the concept of demons so probably
you have it maybe if you are not
familiar with the world might seem alien
but the concept of demon you're pretty
familiar with like most computers when
you are running them those like the
software that you kind of start and run
like the comments that we have been seen
like you'd like type ls' and then you
are calling the LS command the LS
command execute because you asked it to
execute and then it finishes a lot of
other programs are just running as
background processes and they're just
executing in the background and waiting
for events to happen or enabling some
sort of functionality in your computer
and examples of these processes may be
like your network man like the part of
your computer that is managing the
network or the part of your computer
that is many in the display things like
that you will see that a lot of what is
enabled by demons is usually programs
that end with a D so for instance when
you are setting into a computer the
receiving computer has to have a SS H
daemon and the program is called sshd
and if this program is not running then
there's no way for me to SSH into the
computer
if the program is running then the
program will be listening and when you
ssh that server the some incoming
request is gonna enter the computer the
computers are gonna send it to this
demon that is running in the background
and then the demon is gonna check
whether you have authorization and if so
it's gonna start like some login cell
that you can start securing from and
different noises handle this is somewhat
differently they will use the main EDS
they all have some sort of system demon
that like responds a lot of these
smaller demons in linux which is the one
of us that were choosing for lot of the
examples the tool that i use you're
using is the system d again like first
system demon that's gonna start a lot of
these processes and if you use this
system CTL comment you can kind of check
for the status of different demons you
can check for which ones are running you
can say you can ask the you can ask it
to start processes stop them this is
kind of a once-off operation you can
also enable it and they like disable
them which will tell the system to kind
of run them at boot or like a stop
running the boot if they were enabled
and perhaps more interestingly you can
configure your own system the unit's so
so far kind of all the examples are a
lot of what the computer has to do but
say you want to run a web server like
one solution you can just like every
time you start your computer you could
like open a timox session and then
execute the command but that's not
really the way that kind of your
computer specs demons to be run the way
your computer specs demons to iran is by
using some sort of system the unit is
like a configuration that tell system
lee how to execute this process so an
example of this is the like here's a
very simple example so the what is
happening
we're describing to systemd what needs
to be done for this program to exit this
example is just running like a simple
Python app you can think of it as a web
server like that can be implemented
using some Python web server library and
here we're saying no this is the
description were saying after like this
is important system v has kind of a list
of services that has to start like all
these demons have to be started but
maybe there are dependencies between
these demos or here we're saying no you
should only start this after the network
has been set up because otherwise how
will you even try to configure our
webserver if I cannot listen to a
network port and then we are defining
what users should run this because you
may want to run this as your user or
maybe other user or maybe the boot user
should be running this and then what
command to run and under what dirt tree
and whenever you have this there's can
be kind of small corner cases that you
might have to be back but this is kind
of the core idea and it can be really
useful to automate the process of
running processes in the background a
small side note to this is the fact that
if you just want to run a command every
so often like I'm like in some
periodicity like say every morning I
want to do something in my computer you
could write a demon that just like does
something on the sleeps for a day but
actually like Linux and Mac OS has
already a demon that does this that is
called grandi and cron digas will take
another type of configuration file where
you can say oh I want to run a command
every day at 8 a.m. or I want to run a
command every 5 minutes and it will just
check for this event and execute and
with a lot of things you will find out
there is already demons that have been
configured for that any questions
regarding
so the the question is where this folder
in the computer where all of these these
are so yeah like yes another some of
these configuration files are is in a
couple of different folders depending if
where there are system demons or their
user demos here you can see at the the
first line is where you will place this
for the system demon to recognize that
has been installed but if you just want
to list all the demons that is running
in Linux for example you can just do
systemctl status and that's gonna print
a tree of all the systems and which
which demon was spawned by which other
demon and a lot of them will be spawn
directly by system v the next topic is
going to be file systems in user space
so kind of a quick intro to this is the
fact that whenever you're using a modern
oh yeah sir and whenever you're using a
modern operating system you are not
tight on a specific file system so like
more nurses are highly modular and you
can for example in Linux there are like
different file systems that you can use
and the way this works is because the
kernel which is kind of the the what is
running most of the operating system has
some modules that know how to interact
with with a file system so usually when
you do something like that
fubar this is happening at a user level
and then this is going through to the
kernel level and there is this some kind
of layer here that is checking where
this action is happening to figure out
what file system is under so for example
you will have multiple disk all the
different days have different file
systems so kind of the kernel has to
figure out which file system
operations to use and say in this file
might be in an ext for which is the most
common Linux one then whenever you do -
for the the kernel will hear that and
then it will try to figure out like all
these lips in and ext4 file system and
it will perform the Associated
instruction for creating a file in an
ext4 file system
however kind of we caveat to having a
system like this is right now I cannot
have user code that defines how to
create a file and that might be kind of
useful in some cases say I want to have
a file system that every time someone
creates a file sends me an email so I
can like know that like people are
creating these files here I cannot like
modify the kernel to add this so the
solution to this something called fuse
and fuse is an a way of having file
systems in userspace so what fees will
do is if this file instead of being in
instead of being in ext4 if this file is
in a fuse file systems fuse will forward
this operation to some other part of new
circle that will say oh create this file
and here I can have the part of the code
that sends an email to me say no this
this file has been created and if in
case you want to still create the file
you can forward back the request to do
some more kernel operations it might not
seem really practical but this is kind
of just a theory in practice why this is
useful is because now you can have user
level code that executes arbitrary
actions when you try to perform file
system operations I really interesting
example of this is called sshfs
so SSA to fast what does is whenever you
try analysis sshfs fuse file system
whenever you try to create open read
write to a file instead of trying to do
that to a local file has an SSH
connection that to an remote server so
if I try to create a file here it will
use that SSH connection to forward their
operation to the remote system and then
it will perform it there so to all my
local computer to the rest of the
programs running in my computer there is
this path that looks kind of there is
here but all the operations that are
performed to the top are kind of
forwarded to the remote filesystem and
with this idea you will there like some
examples in the notes and you will find
more online of ways people have
leveraged this capability to do fairly
interesting file systems so for example
if instead of having SSH you don't care
about the society because you use like
Dropbox or Google Drive it's fine people
have implemented use file systems that
will mount locally and every time you
try to do an operation locally actually
goes to one of these cloud storage
providers so you can also use something
like Amazon s3 or like Google Cloud
storage that not have like the same kind
of UI system that we kind of synchronize
as Dropbox or word Drive other another
application of this kind of that is not
related to something remotely is
something like an encrypted file system
you may have a file system that every
time you try to write to a file you will
try to write it in plain text but it
will capture their operation it will
encrypt on the goal and then it will
save it as a regular file in your file
system but that's actually encrypted and
once you dismount the file system once
you kind of remove the fuse connection
all that is left in your computer are
just like regular files are encrypted
the last topic I wanna cover is backups
and how you like some good practices
about them the kind of the main idea is
that for every file that you care about
if you don't have a backup
of the file you have like a backup story
of that file you can pretty much loose
it at any moment there are like many
different failures nails one of them is
just how our failure so like your
harddrive can fail at any moment so if
you are just copying making a copy of
your files in the same drive that's not
just like if your hard drive fails the
files are gone the same goes me you have
like an external drive where you are
making a copy but if you are kind of
stirring everything in your home and
your home burns down which yes it's
unlikely but if it happens you just lost
all your data so you'll have some sort
of off-site backup for having this
solution a another kind of thing to take
into account is that synchronization or
mirroring options are not backups so
Google Drive Dropbox that I was
mentioning they will just kind of
propagate whatever is happening in your
computer this goes also for how we're
mirroring like right there just making a
copy if you accidentally delete a file
or someone maliciously deletes or your
file or encrypts then using some
ransomware then you might have a copy
but you have a copy of the same useless
data you you actually have to have a
solution of how you're running your
backups and you should be asking
yourself what actually someone needs to
know last half about you in order to
delete all your data and we have linked
different software's in the in the notes
about how to do this the the last thing
I want to mention about backups is that
a lot of the time when you think about
backups you just think about their local
files and like all my photos and on my
tax return
how can make a backup of that but
increasingly in the modern age there are
more and more web applications and a lot
of data might only live in some cloud
provider like for example if you have
webmail and or not synchronizing it to
your computer it's only living in in
that provider servers and if you don't
have a copy for that and for some reason
you lose access to that account because
you forget your password you get hacked
they think you have violated the Terms
of Service all the data is gone so
you should look into some tools that
people have developed for kind of of
like making offline copies of all the
data so you can make regular backups of
that and kind of that ends these short
section on backups any questions so far
any Drive compiler any moment like they
like we don't like different media have
different kind of rates of failure and
there are like really good statics
online so for example like spinning hard
drives have like a higher rate of
failure than like SSDs for example like
solid-state drives and it's not the case
or like CD driver um but like for if you
draw up a hard drive is like a higher
rate of failure of that failing of
course but in general there's we don't
really have like end-all solution for
saying oh this media is not gonna fail
like pre Matt's like SD cards is these
hard drives CDs degrade with time limit
every data is kind of bound to this
degradation or like this fact that it
could be lost at any moment and use also
another like data can become corrupted
like your disk my look that is okay but
maybe some files were corrupted and
something like synchronization
techniques like Google Drive or Dropbox
we propagate that corruption and by the
time that you realize that things have
to go wrong it's may be too late
all right we're gonna continue this
trend of jumping between random topics
and talk about api's so so far we've
really been talking about how do you do
things more efficiently locally on your
computer like I want to accomplish this
task more efficiently how do I like
configure my editor how do I use my
shell but one thing you should realize
is that very often you can integrate
with the outside world
most services they interact with in your
day to day provide some kind of API for
you to interact with the data that they
store or the services that they provide
and usually those api's are pretty well
documented if you looked at the api's
for things like Facebook or Twitter or
like Google Drive or Gmail many of these
have interfaces that you can interact
with in order to use those services from
your local machine what's really neat is
that you can often combine this with
some of the stuff that we've talked
about in lecture so far like for example
in the data wrangling lecture we looked
at how you can create these pipelines to
extract data from some source that has a
different format than you expected so
for instance the US government has a
free service where you can request the
weather forecast for any given location
in the US and what you do is there is a
URL that you request and if you set the
right parameters and that URL and then
just fetch it what you get back is JSON
which is sort of a well-defined data
format that you can then parse and you
can extract things like your 1414 day
weather forecast and maybe you then pipe
that into your shell and produce some
kind of like handy alias in your
terminal that's just gonna print of some
handy reference for the next 14 days of
weather in whatever location you were in
right these are things you can pretty
easily construct and there's some notes
there are some notes in the notes about
how you might go about this in general
when you're tracked with these api's
you're going to be using URLs of one
form or another and the exact format
varies from service to service but in
general the URL is going to contain some
set of parameters but ultimately you're
just gonna issue a web request to them
and you're gonna get data back in some
format one command you should be aware
of for interacting with these types of
things is one called curl so curl is a
program that you invoke you give it a
URL and it just fetches that URL and
gives you back the response what you do
with that response is entirely up to you
maybe you pipe it through a program like
GQ so sorry JQ so JQ is a JSON query
tool that lets you take in data this
formatted as JSON and then write
a query over it to extract data that
you're interested in and this is one of
the ways in which you can layer these
tools to extract the data that you're
interested in some of these services
also require that you authenticate in
one way or another like for example if
you want to interact with a Facebook API
you need to have some authenticated
token that proves who you are as far as
Facebook is concerned otherwise they
can't say whether you're allowed to say
create a post as a given user very often
these things are gonna use something
called OAuth although not always and you
should look at the documentation for
every service you care about in general
though you will get some kind of secret
token back from the service that you
have to include in the requests that you
make to them either in the URL or in
additional sort of web headers which you
can also send with curl keep in mind
though that these tokens are secret they
are another representation of your user
and anyone who gets their hand on them
can basically pretend to be you they can
do whatever you can do with that token
so keep this in mind don't like stick
them in your dot files and then like
push them onto github that will land you
in trouble you should think of them as a
password
they're also really neat tools online
for integrating services so there's a
service called if this than that which
basically provides integrations with a
bunch of different services and lets you
chain them together and then also access
them partially locally if you wish this
is something that's worth looking into
if there's a particular service you
would like to interact with in a more
efficient manner any questions about
api's alright switching gears entirely
let's talk about command-line arguments
so command-line tools there are a lot of
them and most of them take different
arguments because they do different
things we've talked about looking at man
pages for commands and that will tell
you how this particular command works
what kind of flags and options you might
give to it and what it actually does
when you invoke it but there are some
common themes that are useful to know
about in either in arguments that many
programs take or just concepts the many
of them apply
the first of these we we already
mentioned a little bit in the lecture on
command-line environments which is the -
help flight very often you can pass this
to a program and instead of running it
will just print out information about
how you can run this program often in a
very short condensed way a similar one
is the - - version flag which is prints
the version of the software you're using
this can be really handy if you're doing
something like filing a bug report which
Anita is going to talk a little bit
about later and you want to report what
version you're running on in case the
bug has been fixed since often you can
also do - capital V and that means the
same as version but again check the main
page there's also - verbose or -
lowercase V often which is a flag that
lets you increase the output of the
program it makes the program print more
about what it is doing and very often
you can repeat this flag so you can do
like - V vvvvvv in order to get more
information from that tool and that's me
especially useful if you're trying to
debug a problem if you're like running
our sink and you want to know why did it
decide to copy this file or why did it
decide not to copy this file that kind
of debug output can be useful and often
there's a sort of an inverse flag called
quiet or silent which means that the
tool will not print anything unless it
was like an error anything else we'll
stay quiet about many tools especially
those that do destructive actions or
some kind of action that you cannot undo
provide what's known as a dry run flag
exactly how this is represented the
command line varies from tool to tool
but essentially what this dry run mode
will do is it will run the tool but it
will not actually make any changes
instead it will just inform you of what
it would have done if you hadn't run it
with dry run many of these tools also
have an interactive mode so for example
there are M and move tools both too
often just - I although not always when
you run a tool in interactive mode it
will usually prompt you whenever it's
about to do an action that you can't
undo and it will
of prompt you for a confirmation that it
should actually go ahead when we're
talking about destructive tools
many of them are non-recursive by
default if you try to remove a directory
or you try to operate on a full
directory
they will not continue into the files
inside of that directory
the reason being you might accidentally
like remove your entire hard drive and
that seems bad therefore from any of
these tools they have a recurse flag
often - are but again not always which
lets them traverse down into the tree to
go deeper but you need to opt into this
behavior so this is for example the case
for RM this is also the case for copy in
many tools when they ask you to give a
file name or a path and we talked about
this a little bit in the data wrangling
lecture instead of giving a file name
you can often just give a - just a
single - and what that means is standard
input or standard output depending on
whether that argument is an input file
or an output file this is handy to know
about if you're trying to construct
those kinds of data wrangling pipelines
that we've talked about before many
tools were also default to using
standard in a Cell not if you don't give
any file name at all sometimes you want
to pass something that looks like a flag
or an option to a command but you don't
actually want it to be interpreted as a
flag or an option consider for example
if you wanted to remove a file called -
I what would you do right if you write
the following command RM - I well - I is
a flag - R M so RM would when you run
this command would say tell me what file
to remove you haven't given me a file
and it's because it interprets this as a
flag similarly if you do something like
SSH some machine some command and let's
say - R so this is saying run command
foo on this machine over SSH and it
- pass - r2 foo well the way that both
of these are gonna get interpreted is
that these are flags or in this case
this is a flag but to this command which
is probably not what you expected
actually in the case of SSH it has some
weird special behavior for some of these
but often if you want something to not
be interpreted as a flag there's a very
simple way to opt out of that and that
is using double - if you use double -
what do you tell the command is that
everything following this you should not
interpret so it will not be considered a
flag or an option in the case of RM you
can do this and now RM you will see that
the first argument as a - - and then it
will keep reading arguments but it will
not interpret them as flags so when it
gets the - I it will not interpret it as
the - I flag but just as an argument
similarly for SSH you can do this to
indicate that these are both positional
arguments they are not flags or options
and you should not interpret things that
start with a - no so this is a - - with
a space on both sides yeah any questions
about any of this sort of command line
conventions business alright then let's
talk about window managers so most of
you are used to some kind of
drag-and-drop window manager they
they're like if you're running Windows
or Mac OS or a bun - what comes with the
machine is like there are windows and
they overlap partially on screen and you
can like drag-and-drop and move them
around and resize them and stuff
and that works fine but it is not the
only way to manage windows on your
computer it turns out so what you are
used to is something called a floating
window manager but not all window
managers are floating often you can opt
into other types of window managers that
have different behavior for how they
range your desktop a common alternative
is a tiling window manager so in a
tiling window manager rather than having
floating windows everything is set up
into a tiled layout when you start a
program its window is maximized if you
start another program the original
window shrinks in size and then the new
window takes up some subset of the total
desktop space at no time is your desktop
background visible unless you have no
programs open all of the programs you
have open on any given desktop are going
to share that space this looks a little
bit like T MUX panes like we talked
about earlier where you can sort of
split them in various directions and one
of the reasons why this is handy is it
means you basically never need to go to
your mouse right in order to move
between windows there are keyboard
shortcuts to move different windows
there are keyboard shortcuts for
resizing the windows or swapping them
around on screen and this turns out to
be a pretty efficient way to manage
windows in your computer well I won't go
into too much of detail of what kind of
window managers you might use just know
that these exist out there and they're
worth giving a look they can be a lot
more efficient to work with question of
our window managers all right VPNs
totally related to the previous topic so
VPNs are like all the rage these days
and this makes me really sad it's not
clear the VPNs are all the rage for any
good reason because you shouldn't be
aware of what a VPN does and does not
get you a VPN in the best case is really
just a way for you to change your
internet service provider it's a way for
you to make traffic on the Internet
seem like it's coming from somewhere
else than what you actually are while
that might seem attractive for certain
purposes it's a little unclear what
advice you in terms of security because
all you're really doing is shifting who
you are trusting rather than trusting
who is providing your current internet
service you are trusting that whatever
business is giving you that VPN service
you're trusting that they first of all
have set up this VPN business correctly
but
so that they are not tracking what you
are doing right and it's not clear
whether that change of trust is actually
worth it if you're sitting at some like
dodging dodgy public Wi-Fi network then
maybe but if you're sitting at MIT it's
not clear do you trust your VPN provider
more than you trust like MIT isn t or
maybe you do but but that is a decision
that you need to make about what you
trust who you trust and why you should
also know that much of your traffic
especially the stuff that's on a
sensitive nature on the Internet is
already encrypted whether that's sort of
HTTP or other protocols that use
something like TLS a lot of the
sensitive data is already sent over
encrypted channels and it doesn't really
matter who your network provider is if
you're on a dodgy Wi-Fi network the
stuff that matters is probably encrypted
anyway might not be but if it's not then
your VPN provider can also see it in
plaintext just as much as whoever's
hosts in this dodgy Wi-Fi network and
notice that I said like in the best case
above there are VPN providers who have
been shown to be malicious that do
logging of all your traffic that sell
that traffic to third parties their VPN
providers that have forgotten to enable
encryption on the VPN all of these are
real problems and so you should think
very carefully about whether a VPN
actually serves any good purpose for you
questions about maybe ends yes
so that doesn't that mean so it's a very
good question if you're on a public
Wi-Fi network then the traffic between
you and the wireless access point is not
encrypted at least it's not encrypted
sort of on the outer layer but it might
be encrypted in like HTTPS for example
and it is totally true the people
observing that Wi-Fi network will be
able to see anything that is not
encrypted but the solution to that is to
encrypt all your traffic rather than
necessarily going through a VPN so one
way to do this for example is to use DNS
over TLS or DNS or HTTPS which gives you
a way to actually encrypt even
information that might otherwise leak in
plaintext rather than try to sort of
trust some provider to do that for you
now that said like some in some cases
you might have a trusted institution
that provides a VPN network for you so
for example MIT provides a VPN network
for all MIT students and stuff that you
can sign up to use and in that case you
probably trust MIT more than the other
networks you might be on so it might be
worth it but it's something for you to
think about so DNS is the way that
people turn domain names or your
computer turns domain names into IP
addresses to know what computer to
connect to and that protocol by default
is play is in plain text
there's nothing encrypted about it there
are various ways to encrypt your DNS
traffic some of them are standardized
and some of them are not I won't go into
the exact mechanics here but you should
google it and look at some of the ways
okay the last thing I want to talk about
is markdown so there is a high chance
that some of you are going to write text
over the remaining part of your life and
you will want to mark up that text in
various simple ways and one thing you
could do is start a word or use latex or
something like that to mark up your
documents but that is pretty
heavy-handed approach instead of be nice
if we could just sort of write things
the way we feel like they should be I
don't know how to describe it in a
better way but sort of the natural way
if you want something if you want to put
emphasis on a word you just put like
stars around it or something and then it
just works markdown is essentially that
it is a way to try to encode the way
that we often write text somewhat
naturally into a markup language that
lets you write things like bold text
links lists that sort of stuff in fact
all of the lecture notes for this class
have been written using markdown and
markdown is really very straightforward
the basic rules are in the in the notes
but the basic things you need to know is
in markdown if you put stars around a
word that word is emphasized or some
sequence of words if you put double
stars that word is emphasized strongly
also known as bold there are various
other things you could do like if you
put a dash before a line it is now a
list and it's one list item and you can
amend lists items if you put one dot in
front or some other number it becomes a
numbered list if you put a pound sign in
front of something it becomes a header
like some kind of title header if you
put multiple of them they become
subheadings and you can keep adding more
to these if you want to write code you
can put a single backtick followed by
some code followed by a back tic and now
that is rendered in mono spaced font if
you want multiple lines of code you do a
triple backtick and then code and then
maybe some more code and then triple
backtick and in many cases like if
you're on github for example you can
even type the name of a language up here
after the backticks without a space and
it will be symptoms highlighted in the
language of your choice this is a really
handy thing that is supported in so many
websites nowadays you might not even
realize like in facebook Messenger you
can use many of these they don't
actually officially say they support
markdown anywhere but many of these
things just like sort of happen to work
and it's worth learning at least the
basics and just start using them you can
do links and stuff as well but that's
already in the notes any questions about
markdown right
Anish you're up it's my microphone
working alright this working you guys
hear me in the back the light screen oh
I think I can hear it okay great so
continuing with our theme of random
topics that are all unrelated to the
previous topics we've been talking about
the next thing we're going to talk about
is a program called hammer spoon which
is a tool for doing desktop automation
on Mac OS and I think there's similar
tools for Windows and like so a lot of
the ideas can carry over it you can
google it if you want to figure out how
to do these things on other platforms
but basically hammer spoon is a program
that lets you write Lua scripts the
scripts in a programming language that
interact with various operating system
functionality so you can write code that
interacts with the keyboard and mouse
and connects that to window management
to display management the file system
battery and power management Wi-Fi all
sorts of stuff like basically all the
things that your operating system
manages this tool lets you hook into
those things and so it can let you do
all sorts of neat things by writing just
a couple lines of code just some
examples of cool things you can do with
this tool are you can bind hotkeys to
move windows to specific locations so
demonstration of this is here I have
this window open I press in my
particular setup option command right in
this window moves to the right option
command left this window moves to the
left and I have a couple other shortcuts
for moving things to various places and
so I can kind of have an effect similar
to tiling window managers that John was
talking about earlier
I can move windows to different parts of
my screen to set things up in a
particular way rather than have to use
the mouse to position things where I
want them to be and then like click and
drag to resize windows to the right
shape this is a keyboard track and ooh
the trick but this tool is not limited
to just moving windows around and
binding that to particularly keyboard
shortcuts you can do other things like
create a menu bar button with a bunch of
different options and you can bind those
different options to do different things
so in my particular case I've created
this old menu and then I have a bunch of
different things that I do reasonably
frequently and clicking on these things
invokes a particular Lua function that
I've written that interacts with this
library so for example here this rescue
windows thing is a particular thing
where I often work with multiple
displays and sometimes my operating
system gets confused and I have some
window that and
up kind of like off of my display and
how do i how do I get this thing back
well that's what this whoops
not that that's what this rescue windows
thing does it brings windows that are
off the screen back onto the screen
another neat thing I have setup here is
I have particularly layouts that I've
named said like a door I'm in a lab in a
laptop layout so for example in my lab I
have this screen and I have another
screen and have another screen besides
that in a different orientation and I
have this particular setup that I want
where I want
maybe my terminal full screen on here
and my chat program over here and this
screen split up into five segments with
different programs in different places
here I can when I show up to lava hands
go here and like click layout lab and it
will invoke some code which is not all
that complicated like ten lines of code
describes a particular layout and it
will instantiate that layout and put all
the things where they need to go I could
even in theory automate some of these
things where my computer could figure
out like I plug in a display and my
computer knows oh this is the display
that you have in your lab let me auda
matically instantiate this layout for
you that's another thing you can do with
hammer spoon and other wacky things you
can do like you can do things like it
can detect your Wi-Fi network that
you're on so it knows kind of where you
are maybe I have a different Wi-Fi
network name at home versus in lab and I
can do things like when I show up to lab
automatically mute my speakers so I
don't have like embarrassing music play
out loud in my lab another kind of cool
example is so I have a Mac it has a
fancy power supply and a lot of my
friends have computers that look the
same as this and their power supply
bricks look the same as mine and
sometimes I use their power brick
because I forgot mine at home or
something this tool can actually with
like three or four lines of code do neat
things like show you a warning like
it'll pop up a warning if you've
accidentally taken your friend's power
supply and plugged it into your computer
instead of using your own so too high
level this tool lets you run arbitrary
Lua code and do things like bind it to
menu buttons or key presses and it
interacts with a large part of the
operating system in order to do all
sorts of cool stuff so that is hammer
spoon any questions about that
cool moving on to the next topic
completely unrelated to the previous one
its booting and live USBs so the
operating system on your computer
Windows or Mac OS
or whatever you're used to is not
exactly like the first thing that runs
on your machine when it turns on there's
something else that happens in the boot
process before your operating system is
loaded and there's some interesting
stuff that you can do there so you might
have seen when you turn on your computer
it says something like press f9 to
configure the BIOS or press f12 to enter
the boot menu the particular key
sequences may depend on your machine and
specific configuration but this is a
general pattern and you can configure
all sorts of interesting Hardware
related stuff here so it's worth
checking out and another thing you can
do in this boot menu is you can have
your computer start off from an
alternate boot device so by default like
my laptop here has a solid-state drive
and it boots Mac OS when it turns on but
I can also say plug in a USB flash drive
that has an operating system installed
on the flash drive and then at boot tell
my computer to boot from that flash
drive instead of the built-in
solid-state disk and this is useful for
example if I've broken my operating
system install and I want to do
something like get the data off my
computer or maybe want to fix the
operating system like maybe there's some
critical files somewhere that I've
deleted or I forgot my password I need
to go like tweak some files in order to
reset it booting from a live USB booting
from the separate operating system
that's installed on a flash drive can
let me do that like boot up my operating
system mount the hard disk that's on my
current machine I'm working on and then
go make some tweaks or copy data off of
that and so live VSPs are really useful
and in the lecture notes we've linked to
a tool that can help you create them
really easily any questions about the
boot process or live USBs all right next
topic is virtual machines vagrant docker
the cloud and OpenStack I think last
year we had an entire lecture on this
topic this year we're going to condense
it into like one minute so at a high
level virtual machines and similar tools
like containers let you emulate a whole
computer system within your current
machine so like I'm running Mac OS here
but within my Mac OS environment I can
simulate a machine that's running say
Ubuntu or some other operating system
and this is a nice way of creating an
isolated environment for testing or for
development or for exploration for
example doing things like running
potentially malicious code that should
be isolated from my current environment
I think the most common use case for
programmers is to use virtual machines
are containers to create development
environments so I'm using Mac OS and I
have some
of services and stuff and libraries
installed in my current machine but I
might want for example I'm working on
some web programming project and I want
to run on an Debian machine and I need
Postgres like a database server
installed rather than install that all
on my Mac OS machine I can instantiate
this new machine just for this just for
the development purposes now virtual
machines like that's a general concept
there are a bunch of programs that let
you that are called virtual machine
hypervisors that support this
functionality on your machine and then
there are tools that let you script
these hypervisors in order to specify
machine configurations like operating
system and like what packages you want
installed and what services you want
installed in plain text and so this is
an example on the screen right here and
this is done using a system called
vagrant and you can it's linked in the
lecture notes you can look into this if
you're curious so basically and the
short plain text file I can specify okay
I want a machine that's running Debian
it should have post grass and Redis and
Python and stuff installed on it and
then once I have this configuration I
can just type in vagrant up and what it
does is it reads this file and
instantiates a new machine based on this
configuration and then after I've done
that I can do vagrant SSH to SSH into
this virtual machine so it's not a
remote machine running on some other
piece of hardware somewhere it's just
simulated on my own machine but now here
I have an Ubuntu box like I do now let's
be really stashe it's like or sorry not
a bunch of debian here with all the
things I want installed in here and I
can do my development inside this
isolated environment and not kind of
install as junk on my Mac OS machine now
so that's vagrant there's similar tools
like docker that are conceptually
similar but use containers instead of
virtual machines it's a distinction that
we're not going to talk about in too
much detail right now and so you can run
VMs on your own computer but you can
also rent virtual machines on the cloud
and so it's a nice way to get instant
access to like one example is you might
want to compute it's always on always
connected to the internet and has a
public IP address like maybe you want to
run a web server that's always available
or you want to run some other service
like say a slack bot or something like
that well a virtual machine rented on
the cloud is one nice way to get that
and these are pretty cheap for like a
low capacity machine with a small CP and
small amount of disk space and
you might want to do is get access to a
machine that's really powerful like with
a lot of CPU cores or with a lot of RAM
or with a whole bunch of GPUs for some
specific purpose like Siri doing deep
learning or so you're doing some other
sorts of sensitive computation well
that's another thing you can do with VMs
on the cloud and finally you can get
access to many more machines than you
have physically physical access to like
if I need a thousand machines but only
for two minutes
to do some very parallel tasks that's
something I can easily do with virtual
machines and popular services for doing
this are things like Amazon AWS or
Google cloud and if you're a member of
MIT csail you can also get free v apps
for research purposes using the CL
OpenStack and so this is also linked in
the lecture notes so any questions about
virtual machines or vagrant doc or
anything like that so the question is
when I say I'm running Ubuntu you're
actually in this case it's Debian when
I'm running a bunch of here do I have
like Ubuntu installed on my machine or
what exactly is going on here so
basically what vagrant did for you and I
type vagrant up is because I've
specified I want Debian here it
downloaded Debian from the internet like
set up a disk image for this new machine
installed Debian into that disk image
then we went to installs for all these
programs so like yes this is on my
computer but all of this is just in a
particular file that's a disk image and
then I'm emulating a machine that is
basically completely isolated from my
current machine this is being run as a
process on my current machine does that
does that answer the question
all right any other questions about VMs
great next topic is also going to be a
quick mention so a lot of your
programmers and you're used to writing
programs and like a tool called
like a tool like vim or some other
editor that you're comfortable with
another thing that can be really neat to
use for particular tasks is something
called a notebook programming
environment and so this is a more
interactive way of writing programs here
on the screen I have a demo this is
something called Jupiter notebook and it
can be used for writing Python programs
I think they also support some other
languages and basically this is a nice
moving interactive programming so
normally you're used to writing a big
program in a file or a collection of
files and once you're done writing it
you can just run the whole program but
this lets you be a little bit more
flexible and
little snippets of code at a time like
for example I can break my program into
these little pieces it's just some
random code I wrote and I can say
execute this cell and I press a
particular key combination to execute
the cell but then I can go back and
tweak my program a little bit like say I
want to have this be a lowercase instead
and then I can execute this cell and
then go and evaluate this thing and this
way I can kind of run little snippets of
code within a Python environment and
it's a nice way of building up programs
piece by piece rather than having to
write everything at once this is really
useful for particular research purposes
like I think a lot of people use these
fridging machine learning work for
example any questions about the idea of
notebook programming environments
they're worth checking out oh so the
question is this looks like it's online
is there an offline version of Jupiter
notebooks so actually this is a thing
that runs in the browser but it's
running locally so I don't know if you
can see it on the screen because it's
kind of small but up here it says
localhost colon 8080 8 8 here I have
running on my own local machine a
Jupiter notebook and it just they've
just built it so it runs within the web
browser that being said there are also
online jupiter notebooks that you can
use where the Python kernel is actually
running on some remote machine you might
want to do this for example like on my
laptop I don't have a fancy GPU but in
my room I have a machine with a fancy
GPU and so when I'm doing machine
learning work I often SSH into that
machine run a jupiter notebook on there
and then open up the interface in my
local web browser so i have access to
that powerful GPU running on my
different machine any other questions
great
the final thing we're going to talk
about today is github so we touched on
this a little bit during the version
control lecture but github is one of the
most popular platforms for open-source
software development it hosts source
code it hosts git repositories but they
also have other tools for managing a
project so and there and like a lot of
the tools we've talked about in this
class are hosted on github for example
like cameras window thing we just talked
about is developed on github and it's
really easy to get started it's
contributing
to open-source projects on github to
help improve the tools that you use
every day and there are two primary ways
you can contribute to projects on github
so let's open up some repository we can
actually go to the github repository for
the class website so this is an
open-source software project and let's
zoom in a little bit so the two ways you
can contribute to projects on github the
two main ways are through issues and
pull requests so one thing that's
actually really helpful to developers
and also pretty lightweight and easy for
users to do is to report issues with a
software project like say you're using
somebody's program you encounter some
bug writing a high quality issues
actually super helpful to developers and
hopefully doesn't take you too much time
and so you can go to here like find the
project on github go to the issues page
and click on new issue and then write
some high quality bug report and then
hopefully the developer will respond and
fix the issue for you
so for example for this class like one
of the students in this class pointed
out an issue with our lecture notes and
after she pointed it out I said okay
like that looks like a reasonable thing
let's fix it and in this particular case
instead of fixing it myself I actually
asked this person do they just want to
fix it for me and so that leads into the
other thing I want to talk about so
issues and pull requests so pull
requests are the second way to
contribute to projects on github and
this involves actually contributing code
back to the project and so if we look at
the pull request for this particular
project you'll see that a bunch of
people have submitted code changes and
the process for doing so so this is
showing the difference the patch that
this person is submitted
basically the process for creating Pleau
requests is a little bit more involved
in submitting issues like you're not
just submitting text you're actually
going to modify their source code and so
we've linked to some guides that explain
the process in a little bit more detail
but at a high level what you do is you
take the repository on github for kit
and then download it locally so now have
your own local copy then you can go and
work on it and do some development work
and fix a bug or add a feature and then
eventually you send what's called a pull
request back to the original developer
so you say here I've made some changes
can you please incorporate them back
into the original project and after that
point what usually happens with these
projects is that the maintainer will go
back and forth with you giving you
feedback on the changes you've proposed
and eventually once everybody's happy
they will merge in your changes and
they'll be available to everybody uses
the project so that is how you can
contribute to projects on github and
make software better for everybody
and so any questions about get up cool
ok so that is it for the topics for
today any questions about the lecture
overall great okay so before before we
finish a quick description about
tomorrow's lecture so today was all the
topics we thought are interesting we
should talk about tomorrow's lecture is
gonna be about all the topics you think
are interesting and that we should talk
about so tomorrow is going to be a Q&A
lecture and after today after the
lecture will submit where you will email
out a link where you can submit
questions for us to answer and so please
go and fill that out otherwise we won't
have too much to talk about tomorrow
great so hopefully see you tomorrow in
our Q&A lecture